{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Selecting Subsets of Data in Pandas\n",
    "\n",
    "This notebook is also available as a [blog post on Medium](https://medium.com/dunder-data/selecting-subsets-of-data-in-pandas-39e811c81a0c).\n",
    "\n",
    "## Part 2: Boolean Indexing\n",
    "This is part 2 of a seven-part series on how to select subsets of data from a pandas DataFrame or Series. Pandas offers a wide variety of options for subset selection which necessitates multiple articles. This series is broken down into the following 7 topics.\n",
    "\n",
    "1. Selection with `[]`, `.loc` and `.iloc`\n",
    "1. Boolean indexing\n",
    "1. Assigning subsets of data\n",
    "1. Selection with a MultiIndex\n",
    "1. Selecting subsets of data with methods\n",
    "1. Selections with other Index types\n",
    "1. Internals, Miscellaneous, and Conclusion\n",
    "\n",
    "## Part 1 vs Part 2 subset selection\n",
    "\n",
    "Part 1 of this series covered subset selection with `[]`, `.loc` and `.iloc`. All three of these **indexers** use either the row/column labels or their integer location to make selections. The actual **data** of the Series/DataFrame is not used at all during the selection. \n",
    "\n",
    "In Part 2 of this series, on **boolean indexing**, we will select subsets of data based on the actual values of the data in the Series/DataFrame and NOT on their row/column labels or integer locations. \n",
    "\n",
    "\n",
    "## Documentation on boolean selection\n",
    "I will always recommend reading the official documentation in addition to this tutorial when learning about boolean selection. The documentation uses more formal examples with dummy data, but is still an excellent resource.\n",
    "\n",
    "The documentation use the term **boolean indexing** but you will also see **boolean selection**.\n",
    "\n",
    "[Boolean Indexing from pandas documentation](http://pandas.pydata.org/pandas-docs/stable/indexing.html#boolean-indexing)\n",
    "\n",
    "## Stack Overflow Data\n",
    "\n",
    "The data that we will use for this tutorial comes from [Stack Overflow's data explorer](https://data.stackexchange.com/stackoverflow/query/new), which is a fantastic tool to gather an incredible amount of data from the site. You must know SQL in order to use the data explorer. The data explorer allows you to save queries. [Take a look at the query](http://data.stackexchange.com/stackoverflow/query/768430/get-all-questions-and-answerers-from-tag) I used to collect the data. \n",
    "\n",
    "The table below contains data on each question asked on stack overflow [tagged as pandas](https://stackoverflow.com/questions/tagged/pandas). \n",
    "\n",
    "The first question was asked March 30, 2011. Since then, more than 56,000 questions have been added as of December 2, 2017."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5486226</td>\n",
       "      <td>2011-03-30 12:26:50</td>\n",
       "      <td>4</td>\n",
       "      <td>2113</td>\n",
       "      <td>Rolling median in python</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>yueerhu</td>\n",
       "      <td>125.0</td>\n",
       "      <td>Mike Pennington</td>\n",
       "      <td>26995.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5515021</td>\n",
       "      <td>2011-04-01 14:50:44</td>\n",
       "      <td>8</td>\n",
       "      <td>7015</td>\n",
       "      <td>Compute a compounded return series in Python</td>\n",
       "      <td>3</td>\n",
       "      <td>6</td>\n",
       "      <td>7.0</td>\n",
       "      <td>Jason Strimpel</td>\n",
       "      <td>3301.0</td>\n",
       "      <td>Mike Pennington</td>\n",
       "      <td>26995.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5558607</td>\n",
       "      <td>2011-04-05 21:13:50</td>\n",
       "      <td>2</td>\n",
       "      <td>7392</td>\n",
       "      <td>Sort a pandas DataMatrix in ascending order</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Jason Strimpel</td>\n",
       "      <td>3301.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6467832</td>\n",
       "      <td>2011-06-24 12:31:45</td>\n",
       "      <td>9</td>\n",
       "      <td>13056</td>\n",
       "      <td>How to get the correlation between two timeser...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>user814005</td>\n",
       "      <td>117.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7577546</td>\n",
       "      <td>2011-09-28 01:58:38</td>\n",
       "      <td>9</td>\n",
       "      <td>2488</td>\n",
       "      <td>Using pandas, how do I subsample a large DataF...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Uri Laserson</td>\n",
       "      <td>958.0</td>\n",
       "      <td>HYRY</td>\n",
       "      <td>54137.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id         creationdate  score  viewcount  \\\n",
       "0  5486226  2011-03-30 12:26:50      4       2113   \n",
       "1  5515021  2011-04-01 14:50:44      8       7015   \n",
       "2  5558607  2011-04-05 21:13:50      2       7392   \n",
       "3  6467832  2011-06-24 12:31:45      9      13056   \n",
       "4  7577546  2011-09-28 01:58:38      9       2488   \n",
       "\n",
       "                                               title  answercount  \\\n",
       "0                           Rolling median in python            3   \n",
       "1       Compute a compounded return series in Python            3   \n",
       "2        Sort a pandas DataMatrix in ascending order            2   \n",
       "3  How to get the correlation between two timeser...            1   \n",
       "4  Using pandas, how do I subsample a large DataF...            1   \n",
       "\n",
       "   commentcount  favoritecount      quest_name  quest_rep         ans_name  \\\n",
       "0             4            1.0         yueerhu      125.0  Mike Pennington   \n",
       "1             6            7.0  Jason Strimpel     3301.0  Mike Pennington   \n",
       "2             0            1.0  Jason Strimpel     3301.0     Wes McKinney   \n",
       "3             0            7.0      user814005      117.0     Wes McKinney   \n",
       "4             0            5.0    Uri Laserson      958.0             HYRY   \n",
       "\n",
       "   ans_rep  \n",
       "0  26995.0  \n",
       "1  26995.0  \n",
       "2  43310.0  \n",
       "3  43310.0  \n",
       "4  54137.0  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so = pd.read_csv('../../data/stackoverflow_qa.csv')\n",
    "so.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Asking simple questions in plain English\n",
    "\n",
    "Before we get to the technical definition of boolean indexing, let's see some examples of the types of questions it can answer.\n",
    "\n",
    "* Find all questions that were created before 2014\n",
    "* Find all questions with a score more than 50\n",
    "* Find all questions with a score between 50 and 100\n",
    "* Find all questions answered by Scott Boston\n",
    "* Find all questions answered by the following 5 users\n",
    "* Find all questions that were created between March, 2014 and October 2014 that were answered by Unutbu and have score less than 5.\n",
    "* Find all questions that have score between 5 and 10 or have a view count of greater than 10,000\n",
    "* Find all questions that are not answered by Scott Boston\n",
    "\n",
    "You will also see examples like this referred to by the term **queries**.\n",
    "\n",
    "## All queries have criteria\n",
    "Each of the above queries have a strict logical criteria that must be checked one row at a time.\n",
    "\n",
    "## Keep or Discard entire row of data\n",
    "If you were to manually answer the above queries, you would need to scan each row and determine whether the row as a whole meets the criterion or not. If the row meets the criteria, then it is kept and if not, then it is discarded.\n",
    "\n",
    "## Each row will have a `True` or `False` value associated with it\n",
    "When you perform boolean indexing, each row of the DataFrame (or value of a Series) will have a `True` or `False` value associated with it depending on whether or not it meets the criterion. True/False values are known as **boolean**. The documentation refers to the entire procedure as **boolean indexing**. \n",
    "\n",
    "Since we are using the booleans to select data, it is sometimes referred to as **boolean selection**. Essentially, we are using booleans to select subsets of data.\n",
    "\n",
    "## Using `[]` and `.loc` for boolean selection\n",
    "We will use the same three indexers, **`[]`** and **`.loc`** from part 1 to complete our boolean selections. We will do so by placing a sequence of booleans inside of these indexer. The sequence will be the same number of rows/values as the DataFrame/Series it is doing the selection on.\n",
    "\n",
    "The **`.iloc`** indexer can be made to work with boolean selection but is almost never used. A small section towards the end will show why it's unnecessary.\n",
    "\n",
    "## Focus on `[]` for now\n",
    "To simplify things, we will only the brackets, **`[]`**, which I called **just the indexing operator** from part 1. We will get to the other indexers a bit later.\n",
    "\n",
    "## Use a small DataFrame to get started\n",
    "Before we make our first boolean selection, let's simplify matters and use the first five rows of the stack overflow data as our starting DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5486226</td>\n",
       "      <td>2011-03-30 12:26:50</td>\n",
       "      <td>4</td>\n",
       "      <td>2113</td>\n",
       "      <td>Rolling median in python</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>yueerhu</td>\n",
       "      <td>125.0</td>\n",
       "      <td>Mike Pennington</td>\n",
       "      <td>26995.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5515021</td>\n",
       "      <td>2011-04-01 14:50:44</td>\n",
       "      <td>8</td>\n",
       "      <td>7015</td>\n",
       "      <td>Compute a compounded return series in Python</td>\n",
       "      <td>3</td>\n",
       "      <td>6</td>\n",
       "      <td>7.0</td>\n",
       "      <td>Jason Strimpel</td>\n",
       "      <td>3301.0</td>\n",
       "      <td>Mike Pennington</td>\n",
       "      <td>26995.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5558607</td>\n",
       "      <td>2011-04-05 21:13:50</td>\n",
       "      <td>2</td>\n",
       "      <td>7392</td>\n",
       "      <td>Sort a pandas DataMatrix in ascending order</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Jason Strimpel</td>\n",
       "      <td>3301.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6467832</td>\n",
       "      <td>2011-06-24 12:31:45</td>\n",
       "      <td>9</td>\n",
       "      <td>13056</td>\n",
       "      <td>How to get the correlation between two timeser...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>user814005</td>\n",
       "      <td>117.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7577546</td>\n",
       "      <td>2011-09-28 01:58:38</td>\n",
       "      <td>9</td>\n",
       "      <td>2488</td>\n",
       "      <td>Using pandas, how do I subsample a large DataF...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Uri Laserson</td>\n",
       "      <td>958.0</td>\n",
       "      <td>HYRY</td>\n",
       "      <td>54137.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id         creationdate  score  viewcount  \\\n",
       "0  5486226  2011-03-30 12:26:50      4       2113   \n",
       "1  5515021  2011-04-01 14:50:44      8       7015   \n",
       "2  5558607  2011-04-05 21:13:50      2       7392   \n",
       "3  6467832  2011-06-24 12:31:45      9      13056   \n",
       "4  7577546  2011-09-28 01:58:38      9       2488   \n",
       "\n",
       "                                               title  answercount  \\\n",
       "0                           Rolling median in python            3   \n",
       "1       Compute a compounded return series in Python            3   \n",
       "2        Sort a pandas DataMatrix in ascending order            2   \n",
       "3  How to get the correlation between two timeser...            1   \n",
       "4  Using pandas, how do I subsample a large DataF...            1   \n",
       "\n",
       "   commentcount  favoritecount      quest_name  quest_rep         ans_name  \\\n",
       "0             4            1.0         yueerhu      125.0  Mike Pennington   \n",
       "1             6            7.0  Jason Strimpel     3301.0  Mike Pennington   \n",
       "2             0            1.0  Jason Strimpel     3301.0     Wes McKinney   \n",
       "3             0            7.0      user814005      117.0     Wes McKinney   \n",
       "4             0            5.0    Uri Laserson      958.0             HYRY   \n",
       "\n",
       "   ans_rep  \n",
       "0  26995.0  \n",
       "1  26995.0  \n",
       "2  43310.0  \n",
       "3  43310.0  \n",
       "4  54137.0  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so_head = so.head()\n",
    "so_head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Manually create a list of booleans\n",
    "For our first boolean selection, we will not answer any interesting 'English' queries and instead just select rows with a list of booleans.\n",
    "\n",
    "For instance, let's select the first and third rows by creating the following list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "criteria = [True, False, True, False, False]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can pass this list of booleans to just the indexing operator and complete our selection:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5486226</td>\n",
       "      <td>2011-03-30 12:26:50</td>\n",
       "      <td>4</td>\n",
       "      <td>2113</td>\n",
       "      <td>Rolling median in python</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>yueerhu</td>\n",
       "      <td>125.0</td>\n",
       "      <td>Mike Pennington</td>\n",
       "      <td>26995.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5558607</td>\n",
       "      <td>2011-04-05 21:13:50</td>\n",
       "      <td>2</td>\n",
       "      <td>7392</td>\n",
       "      <td>Sort a pandas DataMatrix in ascending order</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Jason Strimpel</td>\n",
       "      <td>3301.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id         creationdate  score  viewcount  \\\n",
       "0  5486226  2011-03-30 12:26:50      4       2113   \n",
       "2  5558607  2011-04-05 21:13:50      2       7392   \n",
       "\n",
       "                                         title  answercount  commentcount  \\\n",
       "0                     Rolling median in python            3             4   \n",
       "2  Sort a pandas DataMatrix in ascending order            2             0   \n",
       "\n",
       "   favoritecount      quest_name  quest_rep         ans_name  ans_rep  \n",
       "0            1.0         yueerhu      125.0  Mike Pennington  26995.0  \n",
       "2            1.0  Jason Strimpel     3301.0     Wes McKinney  43310.0  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so_head[criteria]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Wait a second... Isn't `[]` just for column selection?\n",
    "The primary purpose of *just the indexing operator* for a DataFrame is to select one or more columns by using either a string or a list of strings. Now, all of a sudden, this example is showing that entire rows are selected with boolean values. This is what makes pandas, unfortunately, one of the most confusing libraries to use. \n",
    "\n",
    "## Operator Overloading\n",
    "*Just the indexing* operator is overloaded. This means, that depending on the inputs, pandas will do something completely different. Here are the rules for the different objects you pass to *just the indexing operator*.\n",
    "* string - return a column as a Series\n",
    "* list of strings - return all those columns as a DataFrame\n",
    "* a slice - select rows (can do both label and integer location - confusing!)\n",
    "* a sequence of booleans - select all rows where **`True`**\n",
    "\n",
    "In summary, primarily *just the indexing operator* selects **columns**, but if you pass it a sequence of booleans it will select all **rows** that are **`True`**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What do you mean by 'sequence'?\n",
    "I keep using the term **sequence of booleans** to refer to the `True/False` values. Technically, the most common built-in [Python sequence](https://docs.python.org/3/library/stdtypes.html#typesseq) types are lists and tuples. In addition to a list, you will most often be using a pandas Series as your 'sequence' of booleans.\n",
    "\n",
    "Let's manually create a boolean Series to select the last three rows of **`so_head`**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2     True\n",
       "3     True\n",
       "4     True\n",
       "dtype: bool"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = pd.Series([False, False, True, True, True])\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5558607</td>\n",
       "      <td>2011-04-05 21:13:50</td>\n",
       "      <td>2</td>\n",
       "      <td>7392</td>\n",
       "      <td>Sort a pandas DataMatrix in ascending order</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Jason Strimpel</td>\n",
       "      <td>3301.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6467832</td>\n",
       "      <td>2011-06-24 12:31:45</td>\n",
       "      <td>9</td>\n",
       "      <td>13056</td>\n",
       "      <td>How to get the correlation between two timeser...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>user814005</td>\n",
       "      <td>117.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7577546</td>\n",
       "      <td>2011-09-28 01:58:38</td>\n",
       "      <td>9</td>\n",
       "      <td>2488</td>\n",
       "      <td>Using pandas, how do I subsample a large DataF...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Uri Laserson</td>\n",
       "      <td>958.0</td>\n",
       "      <td>HYRY</td>\n",
       "      <td>54137.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id         creationdate  score  viewcount  \\\n",
       "2  5558607  2011-04-05 21:13:50      2       7392   \n",
       "3  6467832  2011-06-24 12:31:45      9      13056   \n",
       "4  7577546  2011-09-28 01:58:38      9       2488   \n",
       "\n",
       "                                               title  answercount  \\\n",
       "2        Sort a pandas DataMatrix in ascending order            2   \n",
       "3  How to get the correlation between two timeser...            1   \n",
       "4  Using pandas, how do I subsample a large DataF...            1   \n",
       "\n",
       "   commentcount  favoritecount      quest_name  quest_rep      ans_name  \\\n",
       "2             0            1.0  Jason Strimpel     3301.0  Wes McKinney   \n",
       "3             0            7.0      user814005      117.0  Wes McKinney   \n",
       "4             0            5.0    Uri Laserson      958.0          HYRY   \n",
       "\n",
       "   ans_rep  \n",
       "2  43310.0  \n",
       "3  43310.0  \n",
       "4  54137.0  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so_head[s]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Take care when creating a boolean Series by hand\n",
    "The above example only worked because the index of both the boolean Series and **`so_head`** were the exact same. Let's output them so you can clearly see this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=5, step=1)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=5, step=1)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so_head.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Boolean selection fails when the index doesn't align\n",
    "When you are using a boolean Series to do boolean selection, the index of both objects must be the exact same. Let's create a slightly different Series with a different index than the DataFrame it is indexing on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2    False\n",
       "3    False\n",
       "4     True\n",
       "5     True\n",
       "6     True\n",
       "dtype: bool"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = pd.Series([False, False, True, True, True], index=[2, 3, 4, 5, 6])\n",
    "s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/Ted/anaconda3/lib/python3.6/site-packages/ipykernel_launcher.py:1: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "ename": "IndexingError",
     "evalue": "Unalignable boolean Series provided as indexer (index of the boolean Series and of the indexed object do not match",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mIndexingError\u001b[0m                             Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-11-43ec5b4aca97>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mso_head\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0ms\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/pandas/core/frame.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m   2131\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mSeries\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mndarray\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mIndex\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlist\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2132\u001b[0m             \u001b[0;31m# either boolean or fancy integer index\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2133\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getitem_array\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2134\u001b[0m         \u001b[0;32melif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mDataFrame\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2135\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getitem_frame\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/pandas/core/frame.py\u001b[0m in \u001b[0;36m_getitem_array\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m   2171\u001b[0m             \u001b[0;31m# check_bool_indexer will throw exception if Series key cannot\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2172\u001b[0m             \u001b[0;31m# be reindexed to match DataFrame rows\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2173\u001b[0;31m             \u001b[0mkey\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcheck_bool_indexer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2174\u001b[0m             \u001b[0mindexer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnonzero\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2175\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_take\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mindexer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconvert\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36mcheck_bool_indexer\u001b[0;34m(ax, key)\u001b[0m\n\u001b[1;32m   2021\u001b[0m         \u001b[0mmask\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0misna\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_values\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2022\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mmask\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0many\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2023\u001b[0;31m             raise IndexingError('Unalignable boolean Series provided as '\n\u001b[0m\u001b[1;32m   2024\u001b[0m                                 \u001b[0;34m'indexer (index of the boolean Series and of '\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2025\u001b[0m                                 'the indexed object do not match')\n",
      "\u001b[0;31mIndexingError\u001b[0m: Unalignable boolean Series provided as indexer (index of the boolean Series and of the indexed object do not match"
     ]
    }
   ],
   "source": [
    "so_head[s]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `IndexingError`: Unalignable boolean Series!\n",
    "If the index of both the boolean Series and the object you are doing boolean selection on don't match exactly, you will get the above error. This is one reason, as you will below, why you will almost never create boolean Series by hand like this."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Also use NumPy arrays\n",
    "You can also use NumPy arrays to do boolean selection. NumPy arrays have no index so you won't get the error above, but your array needs to be the same exact length as the object you are doing boolean selection on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5486226</td>\n",
       "      <td>2011-03-30 12:26:50</td>\n",
       "      <td>4</td>\n",
       "      <td>2113</td>\n",
       "      <td>Rolling median in python</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>yueerhu</td>\n",
       "      <td>125.0</td>\n",
       "      <td>Mike Pennington</td>\n",
       "      <td>26995.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id         creationdate  score  viewcount                     title  \\\n",
       "0  5486226  2011-03-30 12:26:50      4       2113  Rolling median in python   \n",
       "\n",
       "   answercount  commentcount  favoritecount quest_name  quest_rep  \\\n",
       "0            3             4            1.0    yueerhu      125.0   \n",
       "\n",
       "          ans_name  ans_rep  \n",
       "0  Mike Pennington  26995.0  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = np.array([True, False, False, False, False])\n",
    "so_head[a]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Never creating boolean Series by hand\n",
    "You will likely never create a boolean Series by hand as was done above. Instead, you will produce them based on the values of your data.\n",
    "\n",
    "## Use the comparison operators to create boolean Series\n",
    "The primary method of creating a Series of booleans is to use one of the six comparison operators: \n",
    "* **`<`**\n",
    "* **`<=`**\n",
    "* **`>`**\n",
    "* **`>=`**\n",
    "* **`==`**\n",
    "* **`!=`** \n",
    "\n",
    "## Use comparison operator with a single column of data\n",
    "You will almost always use the comparison operators on just a single column or Series of data. For instance, let's create a boolean Series from the **`score`** column. Let's determine if the score is at least 10.\n",
    "\n",
    "We select the score column and then test the condition that each value is greater than or equal to 10. Notice that this operations gets applied to each value in the Series. A boolean Series is returned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2    False\n",
       "3    False\n",
       "4    False\n",
       "5    False\n",
       "6     True\n",
       "7     True\n",
       "8     True\n",
       "9    False\n",
       "Name: score, dtype: bool"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "criteria = so['score'] >= 10\n",
    "criteria.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finally making a boolean selection\n",
    "Now that we have our boolean Series stored in the variable **`criteria`**, we can pass this to *just the indexing operator* to select only the rows that have a score of at least 10. \n",
    "\n",
    "We are going to use the entire **`so`** DataFrame for the rest of the tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7776679</td>\n",
       "      <td>2011-10-15 08:21:17</td>\n",
       "      <td>25</td>\n",
       "      <td>28159</td>\n",
       "      <td>append two data frame with pandas</td>\n",
       "      <td>2</td>\n",
       "      <td>7</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Jean-Pat</td>\n",
       "      <td>882.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7813132</td>\n",
       "      <td>2011-10-18 20:16:12</td>\n",
       "      <td>10</td>\n",
       "      <td>18917</td>\n",
       "      <td>Convert array of string (category) to array of...</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>Jean-Pat</td>\n",
       "      <td>882.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>7837722</td>\n",
       "      <td>2011-10-20 14:46:14</td>\n",
       "      <td>201</td>\n",
       "      <td>223746</td>\n",
       "      <td>What is the most efficient way to loop through...</td>\n",
       "      <td>8</td>\n",
       "      <td>3</td>\n",
       "      <td>115.0</td>\n",
       "      <td>Muppet</td>\n",
       "      <td>1563.0</td>\n",
       "      <td>Nick Crawford</td>\n",
       "      <td>2779.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>8916302</td>\n",
       "      <td>2012-01-18 19:41:27</td>\n",
       "      <td>29</td>\n",
       "      <td>20614</td>\n",
       "      <td>selecting across multiple columns with python ...</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>user248237dfsf</td>\n",
       "      <td>19244.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>8991709</td>\n",
       "      <td>2012-01-24 17:59:53</td>\n",
       "      <td>136</td>\n",
       "      <td>16783</td>\n",
       "      <td>Why are pandas merges in python faster than da...</td>\n",
       "      <td>3</td>\n",
       "      <td>16</td>\n",
       "      <td>60.0</td>\n",
       "      <td>Zach</td>\n",
       "      <td>12484.0</td>\n",
       "      <td>Matt Dowle</td>\n",
       "      <td>41275.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id         creationdate  score  viewcount  \\\n",
       "6   7776679  2011-10-15 08:21:17     25      28159   \n",
       "7   7813132  2011-10-18 20:16:12     10      18917   \n",
       "8   7837722  2011-10-20 14:46:14    201     223746   \n",
       "14  8916302  2012-01-18 19:41:27     29      20614   \n",
       "17  8991709  2012-01-24 17:59:53    136      16783   \n",
       "\n",
       "                                                title  answercount  \\\n",
       "6                   append two data frame with pandas            2   \n",
       "7   Convert array of string (category) to array of...            3   \n",
       "8   What is the most efficient way to loop through...            8   \n",
       "14  selecting across multiple columns with python ...            3   \n",
       "17  Why are pandas merges in python faster than da...            3   \n",
       "\n",
       "    commentcount  favoritecount      quest_name  quest_rep       ans_name  \\\n",
       "6              7            4.0        Jean-Pat      882.0   Wes McKinney   \n",
       "7              0            6.0        Jean-Pat      882.0   Wes McKinney   \n",
       "8              3          115.0          Muppet     1563.0  Nick Crawford   \n",
       "14             0           14.0  user248237dfsf    19244.0   Wes McKinney   \n",
       "17            16           60.0            Zach    12484.0     Matt Dowle   \n",
       "\n",
       "    ans_rep  \n",
       "6   43310.0  \n",
       "7   43310.0  \n",
       "8    2779.0  \n",
       "14  43310.0  \n",
       "17  41275.0  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so_score_10_or_more = so[criteria]\n",
    "so_score_10_or_more.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How many rows have a score of at least ten\n",
    "Just by looking at the head of the resulting DataFrame, we don't know how many rows passed our criterion. Let's output the shape of both our original and our resulting DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(56398, 12)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1505, 12)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so_score_10_or_more.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Only about 3% of questions get a score of 10 or more.\n",
    "\n",
    "## Boolean selection in one line\n",
    "Often, you will see boolean selection happen in a single line of code instead of the multiple lines we used above. If the following is confusing for you, then I recommend storing your boolean Series to a variable like I did with **`criteria`** above.\n",
    "\n",
    "It is possible to put the creation of the boolean Series inside of *just the indexing operator* like this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>7776679</td>\n",
       "      <td>2011-10-15 08:21:17</td>\n",
       "      <td>25</td>\n",
       "      <td>28159</td>\n",
       "      <td>append two data frame with pandas</td>\n",
       "      <td>2</td>\n",
       "      <td>7</td>\n",
       "      <td>4.0</td>\n",
       "      <td>Jean-Pat</td>\n",
       "      <td>882.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7813132</td>\n",
       "      <td>2011-10-18 20:16:12</td>\n",
       "      <td>10</td>\n",
       "      <td>18917</td>\n",
       "      <td>Convert array of string (category) to array of...</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>Jean-Pat</td>\n",
       "      <td>882.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>7837722</td>\n",
       "      <td>2011-10-20 14:46:14</td>\n",
       "      <td>201</td>\n",
       "      <td>223746</td>\n",
       "      <td>What is the most efficient way to loop through...</td>\n",
       "      <td>8</td>\n",
       "      <td>3</td>\n",
       "      <td>115.0</td>\n",
       "      <td>Muppet</td>\n",
       "      <td>1563.0</td>\n",
       "      <td>Nick Crawford</td>\n",
       "      <td>2779.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>8916302</td>\n",
       "      <td>2012-01-18 19:41:27</td>\n",
       "      <td>29</td>\n",
       "      <td>20614</td>\n",
       "      <td>selecting across multiple columns with python ...</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>user248237dfsf</td>\n",
       "      <td>19244.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>8991709</td>\n",
       "      <td>2012-01-24 17:59:53</td>\n",
       "      <td>136</td>\n",
       "      <td>16783</td>\n",
       "      <td>Why are pandas merges in python faster than da...</td>\n",
       "      <td>3</td>\n",
       "      <td>16</td>\n",
       "      <td>60.0</td>\n",
       "      <td>Zach</td>\n",
       "      <td>12484.0</td>\n",
       "      <td>Matt Dowle</td>\n",
       "      <td>41275.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id         creationdate  score  viewcount  \\\n",
       "6   7776679  2011-10-15 08:21:17     25      28159   \n",
       "7   7813132  2011-10-18 20:16:12     10      18917   \n",
       "8   7837722  2011-10-20 14:46:14    201     223746   \n",
       "14  8916302  2012-01-18 19:41:27     29      20614   \n",
       "17  8991709  2012-01-24 17:59:53    136      16783   \n",
       "\n",
       "                                                title  answercount  \\\n",
       "6                   append two data frame with pandas            2   \n",
       "7   Convert array of string (category) to array of...            3   \n",
       "8   What is the most efficient way to loop through...            8   \n",
       "14  selecting across multiple columns with python ...            3   \n",
       "17  Why are pandas merges in python faster than da...            3   \n",
       "\n",
       "    commentcount  favoritecount      quest_name  quest_rep       ans_name  \\\n",
       "6              7            4.0        Jean-Pat      882.0   Wes McKinney   \n",
       "7              0            6.0        Jean-Pat      882.0   Wes McKinney   \n",
       "8              3          115.0          Muppet     1563.0  Nick Crawford   \n",
       "14             0           14.0  user248237dfsf    19244.0   Wes McKinney   \n",
       "17            16           60.0            Zach    12484.0     Matt Dowle   \n",
       "\n",
       "    ans_rep  \n",
       "6   43310.0  \n",
       "7   43310.0  \n",
       "8    2779.0  \n",
       "14  43310.0  \n",
       "17  41275.0  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so[so['score'] >= 10].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Single condition expression\n",
    "Our first example tested a single condition (whether the score was 10 or more). Let's test a different single condition and look for all the questions that are answered by **Scott Boston**. The **`ans_name`** variable holds the display names of the people who posted the accepted answer to the question.\n",
    "\n",
    "We use the **`==`** operator to test for equality and again store this result to the variable **`criteria`**. Again, we pass this variable to *just the indexing operator* which completes our selection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>38161</th>\n",
       "      <td>43491342</td>\n",
       "      <td>2017-04-19 09:14:28</td>\n",
       "      <td>4</td>\n",
       "      <td>167</td>\n",
       "      <td>Merging pandas dataframes based on nearest val...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>AkiRoss</td>\n",
       "      <td>3991.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38178</th>\n",
       "      <td>43190850</td>\n",
       "      <td>2017-04-03 17:31:33</td>\n",
       "      <td>1</td>\n",
       "      <td>284</td>\n",
       "      <td>Python Seaborn Plot ValueError</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Ryan</td>\n",
       "      <td>545.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38237</th>\n",
       "      <td>43176052</td>\n",
       "      <td>2017-04-03 03:21:12</td>\n",
       "      <td>2</td>\n",
       "      <td>39</td>\n",
       "      <td>Convert an indexed pandas matrix to a flat dat...</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>alvas</td>\n",
       "      <td>31923.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38246</th>\n",
       "      <td>43209525</td>\n",
       "      <td>2017-04-04 14:03:17</td>\n",
       "      <td>5</td>\n",
       "      <td>131</td>\n",
       "      <td>Pandas: Optimal way to MultiIndex columns</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>sparc_spread</td>\n",
       "      <td>5470.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38275</th>\n",
       "      <td>43211893</td>\n",
       "      <td>2017-04-04 15:45:17</td>\n",
       "      <td>0</td>\n",
       "      <td>38</td>\n",
       "      <td>How to calculate a index series for a event wi...</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>zsljulius</td>\n",
       "      <td>1102.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id         creationdate  score  viewcount  \\\n",
       "38161  43491342  2017-04-19 09:14:28      4        167   \n",
       "38178  43190850  2017-04-03 17:31:33      1        284   \n",
       "38237  43176052  2017-04-03 03:21:12      2         39   \n",
       "38246  43209525  2017-04-04 14:03:17      5        131   \n",
       "38275  43211893  2017-04-04 15:45:17      0         38   \n",
       "\n",
       "                                                   title  answercount  \\\n",
       "38161  Merging pandas dataframes based on nearest val...            1   \n",
       "38178                     Python Seaborn Plot ValueError            2   \n",
       "38237  Convert an indexed pandas matrix to a flat dat...            2   \n",
       "38246          Pandas: Optimal way to MultiIndex columns            2   \n",
       "38275  How to calculate a index series for a event wi...            1   \n",
       "\n",
       "       commentcount  favoritecount    quest_name  quest_rep      ans_name  \\\n",
       "38161             0            NaN       AkiRoss     3991.0  Scott Boston   \n",
       "38178             3            NaN          Ryan      545.0  Scott Boston   \n",
       "38237             0            NaN         alvas    31923.0  Scott Boston   \n",
       "38246             0            0.0  sparc_spread     5470.0  Scott Boston   \n",
       "38275             3            NaN     zsljulius     1102.0  Scott Boston   \n",
       "\n",
       "       ans_rep  \n",
       "38161  23611.0  \n",
       "38178  23611.0  \n",
       "38237  23611.0  \n",
       "38246  23611.0  \n",
       "38275  23611.0  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# step 1 - create boolean Series\n",
    "criteria = so['ans_name'] == 'Scott Boston'\n",
    "\n",
    "# step 2 - do boolean selection\n",
    "so[criteria].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multiple condition expression\n",
    "So far, both our boolean selections have involved a single condition. You can, of course, have as many conditions as you would like. To do so, you will need to combine your boolean expressions using the three logical operators **and**, **or** and **not**.\n",
    "\n",
    "## Use `&`, `|`, `~`\n",
    "Although Python uses the syntax **`and`**, **`or`**, and **`not`**, these will not work when testing multiple conditions with pandas. The details of why this is so will be left for part 7 of the Series.\n",
    "\n",
    "You must use the following operators with pandas:\n",
    "* **`&`** for **and**\n",
    "* **`|`** for **or**\n",
    "* **`~`** for **not**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Our first multiple condition expression\n",
    "Let's find all the questions that have a score of at least 5 and are answered by Scott Boston. To begin, we will create two separate variable to hold each criteria."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "criteria_1 = so['score'] >= 5\n",
    "criteria_2 = so['ans_name'] == 'Scott Boston'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will then use the **and** operator, the ampersand **`&`**, to combine them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "criteria_all = criteria_1 & criteria_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now pass this final criteria to *just the indexing operator*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>38246</th>\n",
       "      <td>43209525</td>\n",
       "      <td>2017-04-04 14:03:17</td>\n",
       "      <td>5</td>\n",
       "      <td>131</td>\n",
       "      <td>Pandas: Optimal way to MultiIndex columns</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>sparc_spread</td>\n",
       "      <td>5470.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38640</th>\n",
       "      <td>42870703</td>\n",
       "      <td>2017-03-18 05:06:22</td>\n",
       "      <td>5</td>\n",
       "      <td>125</td>\n",
       "      <td>Simultaneous operation of groupby and resample...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>S. Naribole</td>\n",
       "      <td>43.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44358</th>\n",
       "      <td>45064916</td>\n",
       "      <td>2017-07-12 18:16:49</td>\n",
       "      <td>5</td>\n",
       "      <td>428</td>\n",
       "      <td>How to find the correlation between a group of...</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>BKS</td>\n",
       "      <td>506.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44814</th>\n",
       "      <td>44877663</td>\n",
       "      <td>2017-07-03 04:07:59</td>\n",
       "      <td>9</td>\n",
       "      <td>1267</td>\n",
       "      <td>Error: float object has no attribute notnull</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Vivian  Tio</td>\n",
       "      <td>181.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52013</th>\n",
       "      <td>47061564</td>\n",
       "      <td>2017-11-01 18:40:36</td>\n",
       "      <td>5</td>\n",
       "      <td>60</td>\n",
       "      <td>How to create strings from dataframe columns e...</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>hernanavella</td>\n",
       "      <td>1890.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id         creationdate  score  viewcount  \\\n",
       "38246  43209525  2017-04-04 14:03:17      5        131   \n",
       "38640  42870703  2017-03-18 05:06:22      5        125   \n",
       "44358  45064916  2017-07-12 18:16:49      5        428   \n",
       "44814  44877663  2017-07-03 04:07:59      9       1267   \n",
       "52013  47061564  2017-11-01 18:40:36      5         60   \n",
       "\n",
       "                                                   title  answercount  \\\n",
       "38246          Pandas: Optimal way to MultiIndex columns            2   \n",
       "38640  Simultaneous operation of groupby and resample...            1   \n",
       "44358  How to find the correlation between a group of...            2   \n",
       "44814       Error: float object has no attribute notnull            3   \n",
       "52013  How to create strings from dataframe columns e...            5   \n",
       "\n",
       "       commentcount  favoritecount    quest_name  quest_rep      ans_name  \\\n",
       "38246             0            0.0  sparc_spread     5470.0  Scott Boston   \n",
       "38640             0            1.0   S. Naribole       43.0  Scott Boston   \n",
       "44358             5            NaN           BKS      506.0  Scott Boston   \n",
       "44814             2            1.0   Vivian  Tio      181.0  Scott Boston   \n",
       "52013             0            NaN  hernanavella     1890.0  Scott Boston   \n",
       "\n",
       "       ans_rep  \n",
       "38246  23611.0  \n",
       "38640  23611.0  \n",
       "44358  23611.0  \n",
       "44814  23611.0  \n",
       "52013  23611.0  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so[criteria_all]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multiple conditions in one line\n",
    "It is possible to combine the entire expression into a single line. Many pandas users like doing this, others hate it. Regardless, it is a good idea to know how to do so as you will definitely encounter it.\n",
    "\n",
    "## Use parentheses to separate conditions\n",
    "You must encapsulate each condition in a set of parentheses in order to make this work. This again, will be explained in part 7.\n",
    "\n",
    "Each condition will be separated like this:\n",
    "```Python\n",
    "(so['score'] >= 5) & (so['ans_name'] == 'Scott Boston')\n",
    "```\n",
    "\n",
    "We can then drop this expression inside of *just the indexing operator*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>38246</th>\n",
       "      <td>43209525</td>\n",
       "      <td>2017-04-04 14:03:17</td>\n",
       "      <td>5</td>\n",
       "      <td>131</td>\n",
       "      <td>Pandas: Optimal way to MultiIndex columns</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>sparc_spread</td>\n",
       "      <td>5470.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38640</th>\n",
       "      <td>42870703</td>\n",
       "      <td>2017-03-18 05:06:22</td>\n",
       "      <td>5</td>\n",
       "      <td>125</td>\n",
       "      <td>Simultaneous operation of groupby and resample...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>S. Naribole</td>\n",
       "      <td>43.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44358</th>\n",
       "      <td>45064916</td>\n",
       "      <td>2017-07-12 18:16:49</td>\n",
       "      <td>5</td>\n",
       "      <td>428</td>\n",
       "      <td>How to find the correlation between a group of...</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>BKS</td>\n",
       "      <td>506.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44814</th>\n",
       "      <td>44877663</td>\n",
       "      <td>2017-07-03 04:07:59</td>\n",
       "      <td>9</td>\n",
       "      <td>1267</td>\n",
       "      <td>Error: float object has no attribute notnull</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Vivian  Tio</td>\n",
       "      <td>181.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52013</th>\n",
       "      <td>47061564</td>\n",
       "      <td>2017-11-01 18:40:36</td>\n",
       "      <td>5</td>\n",
       "      <td>60</td>\n",
       "      <td>How to create strings from dataframe columns e...</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>hernanavella</td>\n",
       "      <td>1890.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id         creationdate  score  viewcount  \\\n",
       "38246  43209525  2017-04-04 14:03:17      5        131   \n",
       "38640  42870703  2017-03-18 05:06:22      5        125   \n",
       "44358  45064916  2017-07-12 18:16:49      5        428   \n",
       "44814  44877663  2017-07-03 04:07:59      9       1267   \n",
       "52013  47061564  2017-11-01 18:40:36      5         60   \n",
       "\n",
       "                                                   title  answercount  \\\n",
       "38246          Pandas: Optimal way to MultiIndex columns            2   \n",
       "38640  Simultaneous operation of groupby and resample...            1   \n",
       "44358  How to find the correlation between a group of...            2   \n",
       "44814       Error: float object has no attribute notnull            3   \n",
       "52013  How to create strings from dataframe columns e...            5   \n",
       "\n",
       "       commentcount  favoritecount    quest_name  quest_rep      ans_name  \\\n",
       "38246             0            0.0  sparc_spread     5470.0  Scott Boston   \n",
       "38640             0            1.0   S. Naribole       43.0  Scott Boston   \n",
       "44358             5            NaN           BKS      506.0  Scott Boston   \n",
       "44814             2            1.0   Vivian  Tio      181.0  Scott Boston   \n",
       "52013             0            NaN  hernanavella     1890.0  Scott Boston   \n",
       "\n",
       "       ans_rep  \n",
       "38246  23611.0  \n",
       "38640  23611.0  \n",
       "44358  23611.0  \n",
       "44814  23611.0  \n",
       "52013  23611.0  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so[(so['score'] >= 5) & (so['ans_name'] == 'Scott Boston')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using an `or` condition\n",
    "Let's find all the questions that have a score of at least 100 or have at least 10 answers.\n",
    "\n",
    "For the **or** condition, we use the pipe **`|`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>7837722</td>\n",
       "      <td>2011-10-20 14:46:14</td>\n",
       "      <td>201</td>\n",
       "      <td>223746</td>\n",
       "      <td>What is the most efficient way to loop through...</td>\n",
       "      <td>8</td>\n",
       "      <td>3</td>\n",
       "      <td>115.0</td>\n",
       "      <td>Muppet</td>\n",
       "      <td>1563.0</td>\n",
       "      <td>Nick Crawford</td>\n",
       "      <td>2779.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>8991709</td>\n",
       "      <td>2012-01-24 17:59:53</td>\n",
       "      <td>136</td>\n",
       "      <td>16783</td>\n",
       "      <td>Why are pandas merges in python faster than da...</td>\n",
       "      <td>3</td>\n",
       "      <td>16</td>\n",
       "      <td>60.0</td>\n",
       "      <td>Zach</td>\n",
       "      <td>12484.0</td>\n",
       "      <td>Matt Dowle</td>\n",
       "      <td>41275.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>10065051</td>\n",
       "      <td>2012-04-08 18:01:13</td>\n",
       "      <td>83</td>\n",
       "      <td>77902</td>\n",
       "      <td>python-pandas and databases like mysql</td>\n",
       "      <td>11</td>\n",
       "      <td>4</td>\n",
       "      <td>65.0</td>\n",
       "      <td>user1320615</td>\n",
       "      <td>532.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>10373660</td>\n",
       "      <td>2012-04-29 16:10:35</td>\n",
       "      <td>199</td>\n",
       "      <td>207980</td>\n",
       "      <td>Converting a Pandas GroupBy object to DataFrame</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>90.0</td>\n",
       "      <td>saveenr</td>\n",
       "      <td>2421.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>10636024</td>\n",
       "      <td>2012-05-17 12:48:00</td>\n",
       "      <td>36</td>\n",
       "      <td>30386</td>\n",
       "      <td>Python / Pandas - GUI for viewing a DataFrame ...</td>\n",
       "      <td>15</td>\n",
       "      <td>6</td>\n",
       "      <td>27.0</td>\n",
       "      <td>Ross R</td>\n",
       "      <td>1562.0</td>\n",
       "      <td>user1319128</td>\n",
       "      <td>555.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          id         creationdate  score  viewcount  \\\n",
       "8    7837722  2011-10-20 14:46:14    201     223746   \n",
       "17   8991709  2012-01-24 17:59:53    136      16783   \n",
       "60  10065051  2012-04-08 18:01:13     83      77902   \n",
       "75  10373660  2012-04-29 16:10:35    199     207980   \n",
       "99  10636024  2012-05-17 12:48:00     36      30386   \n",
       "\n",
       "                                                title  answercount  \\\n",
       "8   What is the most efficient way to loop through...            8   \n",
       "17  Why are pandas merges in python faster than da...            3   \n",
       "60             python-pandas and databases like mysql           11   \n",
       "75    Converting a Pandas GroupBy object to DataFrame            5   \n",
       "99  Python / Pandas - GUI for viewing a DataFrame ...           15   \n",
       "\n",
       "    commentcount  favoritecount   quest_name  quest_rep       ans_name  \\\n",
       "8              3          115.0       Muppet     1563.0  Nick Crawford   \n",
       "17            16           60.0         Zach    12484.0     Matt Dowle   \n",
       "60             4           65.0  user1320615      532.0            NaN   \n",
       "75             0           90.0      saveenr     2421.0   Wes McKinney   \n",
       "99             6           27.0       Ross R     1562.0    user1319128   \n",
       "\n",
       "    ans_rep  \n",
       "8    2779.0  \n",
       "17  41275.0  \n",
       "60      NaN  \n",
       "75  43310.0  \n",
       "99    555.0  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so[(so['score'] >= 100) | (so['answercount'] >= 10)].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reversing a condition with the `not` operator\n",
    "The tilde character **`~`** represents the **not** operator and reverses a condition. For instance, if we wanted all the questions with score greater than 100, we could do it like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>7837722</td>\n",
       "      <td>2011-10-20 14:46:14</td>\n",
       "      <td>201</td>\n",
       "      <td>223746</td>\n",
       "      <td>What is the most efficient way to loop through...</td>\n",
       "      <td>8</td>\n",
       "      <td>3</td>\n",
       "      <td>115.0</td>\n",
       "      <td>Muppet</td>\n",
       "      <td>1563.0</td>\n",
       "      <td>Nick Crawford</td>\n",
       "      <td>2779.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>8991709</td>\n",
       "      <td>2012-01-24 17:59:53</td>\n",
       "      <td>136</td>\n",
       "      <td>16783</td>\n",
       "      <td>Why are pandas merges in python faster than da...</td>\n",
       "      <td>3</td>\n",
       "      <td>16</td>\n",
       "      <td>60.0</td>\n",
       "      <td>Zach</td>\n",
       "      <td>12484.0</td>\n",
       "      <td>Matt Dowle</td>\n",
       "      <td>41275.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>10373660</td>\n",
       "      <td>2012-04-29 16:10:35</td>\n",
       "      <td>199</td>\n",
       "      <td>207980</td>\n",
       "      <td>Converting a Pandas GroupBy object to DataFrame</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>90.0</td>\n",
       "      <td>saveenr</td>\n",
       "      <td>2421.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>10665889</td>\n",
       "      <td>2012-05-19 14:11:42</td>\n",
       "      <td>144</td>\n",
       "      <td>179896</td>\n",
       "      <td>How to take column-slices of dataframe in pandas</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>65.0</td>\n",
       "      <td>cpa</td>\n",
       "      <td>988.0</td>\n",
       "      <td>Ted Petrou</td>\n",
       "      <td>10426.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106</th>\n",
       "      <td>10715965</td>\n",
       "      <td>2012-05-23 08:12:31</td>\n",
       "      <td>340</td>\n",
       "      <td>408347</td>\n",
       "      <td>add one row in a pandas.DataFrame</td>\n",
       "      <td>15</td>\n",
       "      <td>3</td>\n",
       "      <td>89.0</td>\n",
       "      <td>PhE</td>\n",
       "      <td>1988.0</td>\n",
       "      <td>fred</td>\n",
       "      <td>2342.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           id         creationdate  score  viewcount  \\\n",
       "8     7837722  2011-10-20 14:46:14    201     223746   \n",
       "17    8991709  2012-01-24 17:59:53    136      16783   \n",
       "75   10373660  2012-04-29 16:10:35    199     207980   \n",
       "100  10665889  2012-05-19 14:11:42    144     179896   \n",
       "106  10715965  2012-05-23 08:12:31    340     408347   \n",
       "\n",
       "                                                 title  answercount  \\\n",
       "8    What is the most efficient way to loop through...            8   \n",
       "17   Why are pandas merges in python faster than da...            3   \n",
       "75     Converting a Pandas GroupBy object to DataFrame            5   \n",
       "100   How to take column-slices of dataframe in pandas            7   \n",
       "106                  add one row in a pandas.DataFrame           15   \n",
       "\n",
       "     commentcount  favoritecount quest_name  quest_rep       ans_name  ans_rep  \n",
       "8               3          115.0     Muppet     1563.0  Nick Crawford   2779.0  \n",
       "17             16           60.0       Zach    12484.0     Matt Dowle  41275.0  \n",
       "75              0           90.0    saveenr     2421.0   Wes McKinney  43310.0  \n",
       "100             3           65.0        cpa      988.0     Ted Petrou  10426.0  \n",
       "106             3           89.0        PhE     1988.0           fred   2342.0  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so[~(so['score'] <= 100)].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that there were parentheses around the condition '**`score`** less than equal to 100'. We had to use parentheses here or the operation wouldn't work correctly. \n",
    "\n",
    "Of course, this trivial example has no need for the not operator and can be replaced with the greater than operator, but it's easy to verify.\n",
    "\n",
    "Let's look back up one example and invert the condition of **`score`** at least 100 or number of answers at least 10.  To do this, we will have to wrap our entire expression with parentheses like this:\n",
    "\n",
    "```Python\n",
    "~((so['score'] >= 100) | (so['answercount'] >= 10))\n",
    "```\n",
    "\n",
    "There is a set of parentheses around each inner expression as well."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complex conditions\n",
    "It is possible to build extremely complex conditions to select rows of your DataFrame that meet a very specific criteria. For instance, we can select all questions answered by Scott Boston with **`score`** 5 or more OR questions answered by Ted Petrou with answer count 5 or more.\n",
    "\n",
    "With multiple conditions, its probably best to break out the logic into multiple steps:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>10665889</td>\n",
       "      <td>2012-05-19 14:11:42</td>\n",
       "      <td>144</td>\n",
       "      <td>179896</td>\n",
       "      <td>How to take column-slices of dataframe in pandas</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>65.0</td>\n",
       "      <td>cpa</td>\n",
       "      <td>988.0</td>\n",
       "      <td>Ted Petrou</td>\n",
       "      <td>10426.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38246</th>\n",
       "      <td>43209525</td>\n",
       "      <td>2017-04-04 14:03:17</td>\n",
       "      <td>5</td>\n",
       "      <td>131</td>\n",
       "      <td>Pandas: Optimal way to MultiIndex columns</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>sparc_spread</td>\n",
       "      <td>5470.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38640</th>\n",
       "      <td>42870703</td>\n",
       "      <td>2017-03-18 05:06:22</td>\n",
       "      <td>5</td>\n",
       "      <td>125</td>\n",
       "      <td>Simultaneous operation of groupby and resample...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>S. Naribole</td>\n",
       "      <td>43.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44358</th>\n",
       "      <td>45064916</td>\n",
       "      <td>2017-07-12 18:16:49</td>\n",
       "      <td>5</td>\n",
       "      <td>428</td>\n",
       "      <td>How to find the correlation between a group of...</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>BKS</td>\n",
       "      <td>506.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44814</th>\n",
       "      <td>44877663</td>\n",
       "      <td>2017-07-03 04:07:59</td>\n",
       "      <td>9</td>\n",
       "      <td>1267</td>\n",
       "      <td>Error: float object has no attribute notnull</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Vivian  Tio</td>\n",
       "      <td>181.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52013</th>\n",
       "      <td>47061564</td>\n",
       "      <td>2017-11-01 18:40:36</td>\n",
       "      <td>5</td>\n",
       "      <td>60</td>\n",
       "      <td>How to create strings from dataframe columns e...</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>hernanavella</td>\n",
       "      <td>1890.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id         creationdate  score  viewcount  \\\n",
       "100    10665889  2012-05-19 14:11:42    144     179896   \n",
       "38246  43209525  2017-04-04 14:03:17      5        131   \n",
       "38640  42870703  2017-03-18 05:06:22      5        125   \n",
       "44358  45064916  2017-07-12 18:16:49      5        428   \n",
       "44814  44877663  2017-07-03 04:07:59      9       1267   \n",
       "52013  47061564  2017-11-01 18:40:36      5         60   \n",
       "\n",
       "                                                   title  answercount  \\\n",
       "100     How to take column-slices of dataframe in pandas            7   \n",
       "38246          Pandas: Optimal way to MultiIndex columns            2   \n",
       "38640  Simultaneous operation of groupby and resample...            1   \n",
       "44358  How to find the correlation between a group of...            2   \n",
       "44814       Error: float object has no attribute notnull            3   \n",
       "52013  How to create strings from dataframe columns e...            5   \n",
       "\n",
       "       commentcount  favoritecount    quest_name  quest_rep      ans_name  \\\n",
       "100               3           65.0           cpa      988.0    Ted Petrou   \n",
       "38246             0            0.0  sparc_spread     5470.0  Scott Boston   \n",
       "38640             0            1.0   S. Naribole       43.0  Scott Boston   \n",
       "44358             5            NaN           BKS      506.0  Scott Boston   \n",
       "44814             2            1.0   Vivian  Tio      181.0  Scott Boston   \n",
       "52013             0            NaN  hernanavella     1890.0  Scott Boston   \n",
       "\n",
       "       ans_rep  \n",
       "100    10426.0  \n",
       "38246  23611.0  \n",
       "38640  23611.0  \n",
       "44358  23611.0  \n",
       "44814  23611.0  \n",
       "52013  23611.0  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "criteria_1 = (so['score'] >= 5) & (so['ans_name'] == 'Scott Boston')\n",
    "criteria_2 = (so['answercount'] >= 5) & (so['ans_name'] == 'Ted Petrou')\n",
    "criteria_all = criteria_1 | criteria_2\n",
    "so[criteria_all]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lots of `or` conditions in a single column - use `isin`\n",
    "Occasionally, we will want to test equality in a single column to multiple values. This is most common in string columns. For instance, let's say we wanted to find all the questions answered by Scott Boston, Ted Petrou, MaxU, and unutbu.\n",
    "\n",
    "One way to do this would be with four `or` conditions.\n",
    "\n",
    "```Python\n",
    "criteria = ((so['ans_name'] == 'Scott Boston') | (so['ans_name'] == 'Ted Petrou') | \n",
    "            (so['ans_name'] == 'MaxU') | (so['ans_name'] == 'unutbu'))\n",
    "```\n",
    "\n",
    "An easier way is to use the Series method **`isin`**. Pass it a list of all the items you want to check for equality."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2    False\n",
       "3    False\n",
       "4    False\n",
       "Name: ans_name, dtype: bool"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "criteria = so['ans_name'].isin(['Scott Boston', 'Ted Petrou', 'MaxU', 'unutbu'])\n",
    "criteria.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>10665889</td>\n",
       "      <td>2012-05-19 14:11:42</td>\n",
       "      <td>144</td>\n",
       "      <td>179896</td>\n",
       "      <td>How to take column-slices of dataframe in pandas</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>65.0</td>\n",
       "      <td>cpa</td>\n",
       "      <td>988.0</td>\n",
       "      <td>Ted Petrou</td>\n",
       "      <td>10426.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>574</th>\n",
       "      <td>13385860</td>\n",
       "      <td>2012-11-14 19:25:28</td>\n",
       "      <td>29</td>\n",
       "      <td>31712</td>\n",
       "      <td>How can I remove extra whitespace from strings...</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>mpjan</td>\n",
       "      <td>530.0</td>\n",
       "      <td>unutbu</td>\n",
       "      <td>464745.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>593</th>\n",
       "      <td>13446480</td>\n",
       "      <td>2012-11-19 01:20:07</td>\n",
       "      <td>18</td>\n",
       "      <td>7969</td>\n",
       "      <td>Python Pandas: remove entries based on the num...</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>7.0</td>\n",
       "      <td>sashkello</td>\n",
       "      <td>8714.0</td>\n",
       "      <td>unutbu</td>\n",
       "      <td>464745.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>599</th>\n",
       "      <td>13460889</td>\n",
       "      <td>2012-11-19 19:39:06</td>\n",
       "      <td>8</td>\n",
       "      <td>3168</td>\n",
       "      <td>How to redirect all methods of a contained cla...</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Yariv</td>\n",
       "      <td>5013.0</td>\n",
       "      <td>unutbu</td>\n",
       "      <td>464745.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>639</th>\n",
       "      <td>13653030</td>\n",
       "      <td>2012-11-30 20:54:35</td>\n",
       "      <td>6</td>\n",
       "      <td>11798</td>\n",
       "      <td>How do I Pass a List of Series to a Pandas Dat...</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>rhaskett</td>\n",
       "      <td>323.0</td>\n",
       "      <td>unutbu</td>\n",
       "      <td>464745.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           id         creationdate  score  viewcount  \\\n",
       "100  10665889  2012-05-19 14:11:42    144     179896   \n",
       "574  13385860  2012-11-14 19:25:28     29      31712   \n",
       "593  13446480  2012-11-19 01:20:07     18       7969   \n",
       "599  13460889  2012-11-19 19:39:06      8       3168   \n",
       "639  13653030  2012-11-30 20:54:35      6      11798   \n",
       "\n",
       "                                                 title  answercount  \\\n",
       "100   How to take column-slices of dataframe in pandas            7   \n",
       "574  How can I remove extra whitespace from strings...            6   \n",
       "593  Python Pandas: remove entries based on the num...            4   \n",
       "599  How to redirect all methods of a contained cla...            2   \n",
       "639  How do I Pass a List of Series to a Pandas Dat...            5   \n",
       "\n",
       "     commentcount  favoritecount quest_name  quest_rep    ans_name   ans_rep  \n",
       "100             3           65.0        cpa      988.0  Ted Petrou   10426.0  \n",
       "574             0           17.0      mpjan      530.0      unutbu  464745.0  \n",
       "593             1            7.0  sashkello     8714.0      unutbu  464745.0  \n",
       "599             4            5.0      Yariv     5013.0      unutbu  464745.0  \n",
       "639             1            1.0   rhaskett      323.0      unutbu  464745.0  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so[criteria].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combining `isin` with other criteria\n",
    "You can use the resulting boolean Series from the **`isin`** method in the same way you would from the logical operators. For instance, If we wanted to find all the questions answered by the people above and had score greater than 30 we would do the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4389</th>\n",
       "      <td>21800169</td>\n",
       "      <td>2014-02-15 16:18:11</td>\n",
       "      <td>81</td>\n",
       "      <td>183564</td>\n",
       "      <td>Python Pandas: Get index of rows which column ...</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>I want badges</td>\n",
       "      <td>708.0</td>\n",
       "      <td>unutbu</td>\n",
       "      <td>464745.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5179</th>\n",
       "      <td>21415661</td>\n",
       "      <td>2014-01-28 20:04:04</td>\n",
       "      <td>38</td>\n",
       "      <td>46056</td>\n",
       "      <td>Logic operator for boolean indexing in Pandas</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>16.0</td>\n",
       "      <td>user2988577</td>\n",
       "      <td>647.0</td>\n",
       "      <td>unutbu</td>\n",
       "      <td>464745.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7411</th>\n",
       "      <td>25254016</td>\n",
       "      <td>2014-08-11 23:30:16</td>\n",
       "      <td>73</td>\n",
       "      <td>113632</td>\n",
       "      <td>Pandas - Get first row value of a given column</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>10.0</td>\n",
       "      <td>Ahmed Haque</td>\n",
       "      <td>1363.0</td>\n",
       "      <td>unutbu</td>\n",
       "      <td>464745.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11136</th>\n",
       "      <td>29370057</td>\n",
       "      <td>2015-03-31 13:38:06</td>\n",
       "      <td>51</td>\n",
       "      <td>72821</td>\n",
       "      <td>Select dataframe rows between two dates</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>darkpool</td>\n",
       "      <td>1502.0</td>\n",
       "      <td>unutbu</td>\n",
       "      <td>464745.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25710</th>\n",
       "      <td>38250710</td>\n",
       "      <td>2016-07-07 16:26:26</td>\n",
       "      <td>47</td>\n",
       "      <td>20477</td>\n",
       "      <td>How to split data into 3 sets (train, validati...</td>\n",
       "      <td>3</td>\n",
       "      <td>7</td>\n",
       "      <td>15.0</td>\n",
       "      <td>CentAu</td>\n",
       "      <td>1923.0</td>\n",
       "      <td>MaxU</td>\n",
       "      <td>83732.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id         creationdate  score  viewcount  \\\n",
       "4389   21800169  2014-02-15 16:18:11     81     183564   \n",
       "5179   21415661  2014-01-28 20:04:04     38      46056   \n",
       "7411   25254016  2014-08-11 23:30:16     73     113632   \n",
       "11136  29370057  2015-03-31 13:38:06     51      72821   \n",
       "25710  38250710  2016-07-07 16:26:26     47      20477   \n",
       "\n",
       "                                                   title  answercount  \\\n",
       "4389   Python Pandas: Get index of rows which column ...            2   \n",
       "5179       Logic operator for boolean indexing in Pandas            1   \n",
       "7411      Pandas - Get first row value of a given column            4   \n",
       "11136            Select dataframe rows between two dates            4   \n",
       "25710  How to split data into 3 sets (train, validati...            3   \n",
       "\n",
       "       commentcount  favoritecount     quest_name  quest_rep ans_name  \\\n",
       "4389              0           40.0  I want badges      708.0   unutbu   \n",
       "5179              3           16.0    user2988577      647.0   unutbu   \n",
       "7411              1           10.0    Ahmed Haque     1363.0   unutbu   \n",
       "11136             0           26.0       darkpool     1502.0   unutbu   \n",
       "25710             7           15.0         CentAu     1923.0     MaxU   \n",
       "\n",
       "        ans_rep  \n",
       "4389   464745.0  \n",
       "5179   464745.0  \n",
       "7411   464745.0  \n",
       "11136  464745.0  \n",
       "25710   83732.0  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "criteria_1 = so['ans_name'].isin(['Scott Boston', 'Ted Petrou', 'MaxU', 'unutbu'])\n",
    "criteria_2 = so['score'] > 30\n",
    "criteria_all = criteria_1 & criteria_2\n",
    "so[criteria_all].tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use `isnull` to find rows with missing values\n",
    "The **`isnull`** method returns a boolean Series where True indicates where a missing value is. For instance, questions that do not have an **accepted answer** have missing values for **`ans_name`**. Let's call **`isnull`** on this column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2    False\n",
       "3    False\n",
       "4    False\n",
       "5     True\n",
       "Name: ans_name, dtype: bool"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "no_answer = so['ans_name'].isnull()\n",
    "no_answer.head(6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is just another boolean Series which we can pass to *just the indexing operator*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>7766400</td>\n",
       "      <td>2011-10-14 10:33:54</td>\n",
       "      <td>2</td>\n",
       "      <td>977</td>\n",
       "      <td>Pandas + Django + mod_wsgi + virtualenv</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Evan Davey</td>\n",
       "      <td>100.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>8451327</td>\n",
       "      <td>2011-12-09 20:27:24</td>\n",
       "      <td>1</td>\n",
       "      <td>2435</td>\n",
       "      <td>Python map() function output into Pandas DataF...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>briant57</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>8966871</td>\n",
       "      <td>2012-01-23 03:21:00</td>\n",
       "      <td>0</td>\n",
       "      <td>287</td>\n",
       "      <td>Running Python/Numpy/Pandas on older secure co...</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Casey</td>\n",
       "      <td>16.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>9641916</td>\n",
       "      <td>2012-03-09 22:36:52</td>\n",
       "      <td>7</td>\n",
       "      <td>5519</td>\n",
       "      <td>Python Pandas: can't find numpy.core.multiarra...</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Dylan Cutler</td>\n",
       "      <td>41.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>9647656</td>\n",
       "      <td>2012-03-10 15:35:26</td>\n",
       "      <td>3</td>\n",
       "      <td>986</td>\n",
       "      <td>Pandas dataframe in mixed mode can't serialize...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>David van Coevorden</td>\n",
       "      <td>45.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         id         creationdate  score  viewcount  \\\n",
       "5   7766400  2011-10-14 10:33:54      2        977   \n",
       "11  8451327  2011-12-09 20:27:24      1       2435   \n",
       "16  8966871  2012-01-23 03:21:00      0        287   \n",
       "29  9641916  2012-03-09 22:36:52      7       5519   \n",
       "30  9647656  2012-03-10 15:35:26      3        986   \n",
       "\n",
       "                                                title  answercount  \\\n",
       "5             Pandas + Django + mod_wsgi + virtualenv            1   \n",
       "11  Python map() function output into Pandas DataF...            1   \n",
       "16  Running Python/Numpy/Pandas on older secure co...            0   \n",
       "29  Python Pandas: can't find numpy.core.multiarra...            4   \n",
       "30  Pandas dataframe in mixed mode can't serialize...            1   \n",
       "\n",
       "    commentcount  favoritecount           quest_name  quest_rep ans_name  \\\n",
       "5              2            1.0           Evan Davey      100.0      NaN   \n",
       "11             0            1.0             briant57        6.0      NaN   \n",
       "16             3            NaN                Casey       16.0      NaN   \n",
       "29             2            NaN         Dylan Cutler       41.0      NaN   \n",
       "30             0            NaN  David van Coevorden       45.0      NaN   \n",
       "\n",
       "    ans_rep  \n",
       "5       NaN  \n",
       "11      NaN  \n",
       "16      NaN  \n",
       "29      NaN  \n",
       "30      NaN  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so[no_answer].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An alias of **`isnull`** is the **`isna`** method. Alias means it is the same exact method with a different name."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Boolean Selection on a Series\n",
    "All the examples thus far have taken place on the **`so`** DataFrame. Boolean selection on a Series happens almost identically. Since there is only one dimension of data, the queries you ask are usually going to be simpler.\n",
    "\n",
    "First, let's select a single column of data as a Series such as the **`commentcount`** column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    4\n",
       "1    6\n",
       "2    0\n",
       "3    0\n",
       "4    0\n",
       "Name: commentcount, dtype: int64"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = so['commentcount']\n",
    "s.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's test for number of comments greater than 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2    False\n",
       "3    False\n",
       "4    False\n",
       "Name: commentcount, dtype: bool"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "criteria = s > 10\n",
    "criteria.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that there is no column selection here as we are already down to a single column. Let's pass this criteria to *just the indexing operator* to select just the values greater than 10."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17     16\n",
       "76     14\n",
       "566    11\n",
       "763    12\n",
       "781    19\n",
       "Name: commentcount, dtype: int64"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s[criteria].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could have done this in one step like this"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17     16\n",
       "76     14\n",
       "566    11\n",
       "763    12\n",
       "781    19\n",
       "Name: commentcount, dtype: int64"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s[s > 10].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we wanted to find those comments greater than 10 but less than 15 we could have used an **and** condition like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "76     14\n",
       "566    11\n",
       "763    12\n",
       "787    12\n",
       "837    13\n",
       "Name: commentcount, dtype: int64"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s[(s > 10) & (s < 15)].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Another possibility is the `between` method\n",
    "Pandas has lots of duplicate functionality built in to it. Instead of writing two boolean conditions to select all values inside of a range as was done above, you can use the **`between`** method to create a boolean Series. To use, pass it the left and right end points of the range. These endpoints are inclusive.\n",
    "\n",
    "So, to replicate the previous example, you could have done this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "76     14\n",
       "566    11\n",
       "763    12\n",
       "787    12\n",
       "837    13\n",
       "Name: commentcount, dtype: int64"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s[s.between(11, 14)].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simultaneous boolean selection with rows and column labels with `.loc`\n",
    "The **`.loc`** indexer was thoroughly covered in part 1 and will now be covered here to simultaneously select rows and columns. In part 1, it was stated that **`.loc`** made selections only by **label**. This wasn't strictly true as it is also able to do boolean selection along with selection by label.\n",
    "\n",
    "Remember that **`.loc`** takes both a row selection and a column selection separated by a comma. Since the row selection comes first, you can pass it the same exact inputs that you do for *just the indexing operator* and get the same results.\n",
    "\n",
    "Let's take a look at a couple examples from above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>38246</th>\n",
       "      <td>43209525</td>\n",
       "      <td>2017-04-04 14:03:17</td>\n",
       "      <td>5</td>\n",
       "      <td>131</td>\n",
       "      <td>Pandas: Optimal way to MultiIndex columns</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>sparc_spread</td>\n",
       "      <td>5470.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38640</th>\n",
       "      <td>42870703</td>\n",
       "      <td>2017-03-18 05:06:22</td>\n",
       "      <td>5</td>\n",
       "      <td>125</td>\n",
       "      <td>Simultaneous operation of groupby and resample...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>S. Naribole</td>\n",
       "      <td>43.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44358</th>\n",
       "      <td>45064916</td>\n",
       "      <td>2017-07-12 18:16:49</td>\n",
       "      <td>5</td>\n",
       "      <td>428</td>\n",
       "      <td>How to find the correlation between a group of...</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>BKS</td>\n",
       "      <td>506.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44814</th>\n",
       "      <td>44877663</td>\n",
       "      <td>2017-07-03 04:07:59</td>\n",
       "      <td>9</td>\n",
       "      <td>1267</td>\n",
       "      <td>Error: float object has no attribute notnull</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Vivian  Tio</td>\n",
       "      <td>181.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52013</th>\n",
       "      <td>47061564</td>\n",
       "      <td>2017-11-01 18:40:36</td>\n",
       "      <td>5</td>\n",
       "      <td>60</td>\n",
       "      <td>How to create strings from dataframe columns e...</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>hernanavella</td>\n",
       "      <td>1890.0</td>\n",
       "      <td>Scott Boston</td>\n",
       "      <td>23611.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             id         creationdate  score  viewcount  \\\n",
       "38246  43209525  2017-04-04 14:03:17      5        131   \n",
       "38640  42870703  2017-03-18 05:06:22      5        125   \n",
       "44358  45064916  2017-07-12 18:16:49      5        428   \n",
       "44814  44877663  2017-07-03 04:07:59      9       1267   \n",
       "52013  47061564  2017-11-01 18:40:36      5         60   \n",
       "\n",
       "                                                   title  answercount  \\\n",
       "38246          Pandas: Optimal way to MultiIndex columns            2   \n",
       "38640  Simultaneous operation of groupby and resample...            1   \n",
       "44358  How to find the correlation between a group of...            2   \n",
       "44814       Error: float object has no attribute notnull            3   \n",
       "52013  How to create strings from dataframe columns e...            5   \n",
       "\n",
       "       commentcount  favoritecount    quest_name  quest_rep      ans_name  \\\n",
       "38246             0            0.0  sparc_spread     5470.0  Scott Boston   \n",
       "38640             0            1.0   S. Naribole       43.0  Scott Boston   \n",
       "44358             5            NaN           BKS      506.0  Scott Boston   \n",
       "44814             2            1.0   Vivian  Tio      181.0  Scott Boston   \n",
       "52013             0            NaN  hernanavella     1890.0  Scott Boston   \n",
       "\n",
       "       ans_rep  \n",
       "38246  23611.0  \n",
       "38640  23611.0  \n",
       "44358  23611.0  \n",
       "44814  23611.0  \n",
       "52013  23611.0  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# same as above\n",
    "so.loc[(so['score'] >= 5) & (so['ans_name'] == 'Scott Boston')]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>10665889</td>\n",
       "      <td>2012-05-19 14:11:42</td>\n",
       "      <td>144</td>\n",
       "      <td>179896</td>\n",
       "      <td>How to take column-slices of dataframe in pandas</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>65.0</td>\n",
       "      <td>cpa</td>\n",
       "      <td>988.0</td>\n",
       "      <td>Ted Petrou</td>\n",
       "      <td>10426.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>574</th>\n",
       "      <td>13385860</td>\n",
       "      <td>2012-11-14 19:25:28</td>\n",
       "      <td>29</td>\n",
       "      <td>31712</td>\n",
       "      <td>How can I remove extra whitespace from strings...</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>mpjan</td>\n",
       "      <td>530.0</td>\n",
       "      <td>unutbu</td>\n",
       "      <td>464745.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>593</th>\n",
       "      <td>13446480</td>\n",
       "      <td>2012-11-19 01:20:07</td>\n",
       "      <td>18</td>\n",
       "      <td>7969</td>\n",
       "      <td>Python Pandas: remove entries based on the num...</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>7.0</td>\n",
       "      <td>sashkello</td>\n",
       "      <td>8714.0</td>\n",
       "      <td>unutbu</td>\n",
       "      <td>464745.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>599</th>\n",
       "      <td>13460889</td>\n",
       "      <td>2012-11-19 19:39:06</td>\n",
       "      <td>8</td>\n",
       "      <td>3168</td>\n",
       "      <td>How to redirect all methods of a contained cla...</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Yariv</td>\n",
       "      <td>5013.0</td>\n",
       "      <td>unutbu</td>\n",
       "      <td>464745.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>639</th>\n",
       "      <td>13653030</td>\n",
       "      <td>2012-11-30 20:54:35</td>\n",
       "      <td>6</td>\n",
       "      <td>11798</td>\n",
       "      <td>How do I Pass a List of Series to a Pandas Dat...</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>rhaskett</td>\n",
       "      <td>323.0</td>\n",
       "      <td>unutbu</td>\n",
       "      <td>464745.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           id         creationdate  score  viewcount  \\\n",
       "100  10665889  2012-05-19 14:11:42    144     179896   \n",
       "574  13385860  2012-11-14 19:25:28     29      31712   \n",
       "593  13446480  2012-11-19 01:20:07     18       7969   \n",
       "599  13460889  2012-11-19 19:39:06      8       3168   \n",
       "639  13653030  2012-11-30 20:54:35      6      11798   \n",
       "\n",
       "                                                 title  answercount  \\\n",
       "100   How to take column-slices of dataframe in pandas            7   \n",
       "574  How can I remove extra whitespace from strings...            6   \n",
       "593  Python Pandas: remove entries based on the num...            4   \n",
       "599  How to redirect all methods of a contained cla...            2   \n",
       "639  How do I Pass a List of Series to a Pandas Dat...            5   \n",
       "\n",
       "     commentcount  favoritecount quest_name  quest_rep    ans_name   ans_rep  \n",
       "100             3           65.0        cpa      988.0  Ted Petrou   10426.0  \n",
       "574             0           17.0      mpjan      530.0      unutbu  464745.0  \n",
       "593             1            7.0  sashkello     8714.0      unutbu  464745.0  \n",
       "599             4            5.0      Yariv     5013.0      unutbu  464745.0  \n",
       "639             1            1.0   rhaskett      323.0      unutbu  464745.0  "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# same as above\n",
    "criteria = so['ans_name'].isin(['Scott Boston', 'Ted Petrou', 'MaxU', 'unutbu'])\n",
    "so.loc[criteria].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Separate row and column selection with a comma for `.loc`\n",
    "The great benefit of **`.loc`** is that it allows you to simultaneously do boolean selection along the rows and make column selections by label.\n",
    "\n",
    "For instance, let's say we wanted to find all the questions with more than 20k views but only return the **`creationdate`**, **`viewcount`**, and **`ans_name`** columns. You would do the following."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>creationdate</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>ans_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2011-10-15 08:21:17</td>\n",
       "      <td>28159</td>\n",
       "      <td>Wes McKinney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2011-10-20 14:46:14</td>\n",
       "      <td>223746</td>\n",
       "      <td>Nick Crawford</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>2012-01-18 19:41:27</td>\n",
       "      <td>20614</td>\n",
       "      <td>Wes McKinney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>2012-03-11 06:00:56</td>\n",
       "      <td>39323</td>\n",
       "      <td>huon</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>2012-03-18 12:53:06</td>\n",
       "      <td>43262</td>\n",
       "      <td>Wes McKinney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>2012-04-04 23:17:23</td>\n",
       "      <td>20171</td>\n",
       "      <td>bmu</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>2012-04-08 18:01:13</td>\n",
       "      <td>77902</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>71</th>\n",
       "      <td>2012-04-18 03:59:55</td>\n",
       "      <td>88115</td>\n",
       "      <td>ely</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>2012-04-29 16:10:35</td>\n",
       "      <td>207980</td>\n",
       "      <td>Wes McKinney</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>2012-04-29 22:41:28</td>\n",
       "      <td>26110</td>\n",
       "      <td>andrew cooke</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           creationdate  viewcount       ans_name\n",
       "6   2011-10-15 08:21:17      28159   Wes McKinney\n",
       "8   2011-10-20 14:46:14     223746  Nick Crawford\n",
       "14  2012-01-18 19:41:27      20614   Wes McKinney\n",
       "31  2012-03-11 06:00:56      39323           huon\n",
       "35  2012-03-18 12:53:06      43262   Wes McKinney\n",
       "58  2012-04-04 23:17:23      20171            bmu\n",
       "60  2012-04-08 18:01:13      77902            NaN\n",
       "71  2012-04-18 03:59:55      88115            ely\n",
       "75  2012-04-29 16:10:35     207980   Wes McKinney\n",
       "76  2012-04-29 22:41:28      26110   andrew cooke"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so.loc[so['viewcount'] > 20000, ['creationdate', 'viewcount', 'ans_name']].head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You could have broken each selection into pieces like this:\n",
    "\n",
    "```Python\n",
    "\n",
    "row_selection = so['viewcount'] > 20000\n",
    "col_selection = ['creationdate', 'viewcount', 'ans_name']\n",
    "so.loc[row_selection, col_selection]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lots of combinations possible with `.loc`\n",
    "Remember that **`.loc`** can take a string, a list of strings or a slice. You can use all three possible ways to select your data. You can also make very complex boolean selections for your rows.\n",
    "\n",
    "Let's select rows with **`favoritecount`** between 30 and 40 and every third column beginning from **`title`** to the end."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>ans_name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>348</th>\n",
       "      <td>Pandas: create two new columns in a dataframe ...</td>\n",
       "      <td>39.0</td>\n",
       "      <td>DSM</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>555</th>\n",
       "      <td>How can I replace all the NaN values with Zero...</td>\n",
       "      <td>39.0</td>\n",
       "      <td>Aman</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>643</th>\n",
       "      <td>Pandas DataFrame: remove unwanted parts from s...</td>\n",
       "      <td>31.0</td>\n",
       "      <td>eumiro</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>667</th>\n",
       "      <td>Creating an empty Pandas DataFrame, then filli...</td>\n",
       "      <td>40.0</td>\n",
       "      <td>Andy Hayden</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>892</th>\n",
       "      <td>Apply multiple functions to multiple groupby c...</td>\n",
       "      <td>35.0</td>\n",
       "      <td>Zelazny7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 title  favoritecount  \\\n",
       "348  Pandas: create two new columns in a dataframe ...           39.0   \n",
       "555  How can I replace all the NaN values with Zero...           39.0   \n",
       "643  Pandas DataFrame: remove unwanted parts from s...           31.0   \n",
       "667  Creating an empty Pandas DataFrame, then filli...           40.0   \n",
       "892  Apply multiple functions to multiple groupby c...           35.0   \n",
       "\n",
       "        ans_name  \n",
       "348          DSM  \n",
       "555         Aman  \n",
       "643       eumiro  \n",
       "667  Andy Hayden  \n",
       "892     Zelazny7  "
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# weird but possible\n",
    "so.loc[so['favoritecount'].between(30, 40), 'title'::3].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Boolean selection for the columns?\n",
    "It is actually possible to use a sequence of booleans to select columns. You pass a list, Series, or array of booleans the same length as the number of columns to **`.loc`**.\n",
    "\n",
    "Let's do a simple manual example where we create a list of booleans by hand. First, let's find out how many columns are in our dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(56398, 12)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a list of 12 booleans"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[True,\n",
       " False,\n",
       " False,\n",
       " True,\n",
       " False,\n",
       " False,\n",
       " True,\n",
       " False,\n",
       " False,\n",
       " True,\n",
       " False,\n",
       " False]"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "col_bools = [True, False, False] * 4\n",
    "col_bools"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use **`.loc`** to select all rows with just the `True` columns from **`col_bools`**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>quest_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5486226</td>\n",
       "      <td>2113</td>\n",
       "      <td>4</td>\n",
       "      <td>125.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5515021</td>\n",
       "      <td>7015</td>\n",
       "      <td>6</td>\n",
       "      <td>3301.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5558607</td>\n",
       "      <td>7392</td>\n",
       "      <td>0</td>\n",
       "      <td>3301.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6467832</td>\n",
       "      <td>13056</td>\n",
       "      <td>0</td>\n",
       "      <td>117.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7577546</td>\n",
       "      <td>2488</td>\n",
       "      <td>0</td>\n",
       "      <td>958.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id  viewcount  commentcount  quest_rep\n",
       "0  5486226       2113             4      125.0\n",
       "1  5515021       7015             6     3301.0\n",
       "2  5558607       7392             0     3301.0\n",
       "3  6467832      13056             0      117.0\n",
       "4  7577546       2488             0      958.0"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so.loc[:, col_bools].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can simultaneously select rows and columns too. Let's select the same columns but for rows that have over 500,000 views."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>quest_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>171</th>\n",
       "      <td>11285613</td>\n",
       "      <td>526432</td>\n",
       "      <td>4</td>\n",
       "      <td>3369.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>181</th>\n",
       "      <td>11346283</td>\n",
       "      <td>931604</td>\n",
       "      <td>1</td>\n",
       "      <td>4206.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>397</th>\n",
       "      <td>12555323</td>\n",
       "      <td>698537</td>\n",
       "      <td>0</td>\n",
       "      <td>2920.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>581</th>\n",
       "      <td>13411544</td>\n",
       "      <td>802655</td>\n",
       "      <td>0</td>\n",
       "      <td>8807.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1253</th>\n",
       "      <td>17071871</td>\n",
       "      <td>549481</td>\n",
       "      <td>0</td>\n",
       "      <td>3374.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4587</th>\n",
       "      <td>19482970</td>\n",
       "      <td>541299</td>\n",
       "      <td>1</td>\n",
       "      <td>3483.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            id  viewcount  commentcount  quest_rep\n",
       "171   11285613     526432             4     3369.0\n",
       "181   11346283     931604             1     4206.0\n",
       "397   12555323     698537             0     2920.0\n",
       "581   13411544     802655             0     8807.0\n",
       "1253  17071871     549481             0     3374.0\n",
       "4587  19482970     541299             1     3483.0"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so.loc[so['viewcount'] > 500000, col_bools]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A more practical example\n",
    "Let's see a slightly more practical example of doing boolean selection on the columns. Let's say we flipped 10 coins one-hundred times and store each trial in a column in the DataFrame below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>a</th>\n",
       "      <th>b</th>\n",
       "      <th>c</th>\n",
       "      <th>d</th>\n",
       "      <th>e</th>\n",
       "      <th>f</th>\n",
       "      <th>g</th>\n",
       "      <th>h</th>\n",
       "      <th>i</th>\n",
       "      <th>j</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   a  b  c  d  e  f  g  h  i  j\n",
       "0  0  0  0  1  0  1  0  0  1  0\n",
       "1  1  0  0  1  1  0  0  1  0  0\n",
       "2  0  1  1  0  1  1  0  1  0  0\n",
       "3  1  0  1  0  0  1  1  1  0  0\n",
       "4  0  0  0  1  0  1  0  0  1  1"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coins = pd.DataFrame(np.random.randint(0, 2, (100, 10)), \n",
    "                     columns=list('abcdefghij'))\n",
    "coins.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100, 10)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coins.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we are interested in selecting only the columns that have more than 50% heads, we could first take the mean of each column like this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    0.50\n",
       "b    0.46\n",
       "c    0.48\n",
       "d    0.47\n",
       "e    0.43\n",
       "f    0.52\n",
       "g    0.44\n",
       "h    0.47\n",
       "i    0.57\n",
       "j    0.44\n",
       "dtype: float64"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coin_mean = coins.mean()\n",
    "coin_mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's test the condition that the percentage is greater than .5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "a    False\n",
       "b    False\n",
       "c    False\n",
       "d    False\n",
       "e    False\n",
       "f     True\n",
       "g    False\n",
       "h    False\n",
       "i     True\n",
       "j    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coin_mean > .5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we can use this boolean Series to select only the columns that meet our criteria."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>f</th>\n",
       "      <th>i</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   f  i\n",
       "0  1  1\n",
       "1  0  0\n",
       "2  1  0\n",
       "3  1  0\n",
       "4  1  1"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coins.loc[:, coins.mean() > .5].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Column to column comparisons\n",
    "All of the previous Series comparisons happened against a single scalar value. It is possible to create a boolean Series by comparing one column to another. For instance, we can find all the questions where there are more answers than **`score`**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>8273092</td>\n",
       "      <td>2011-11-25 18:39:02</td>\n",
       "      <td>1</td>\n",
       "      <td>2333</td>\n",
       "      <td>python: pandas install errors</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>codingknob</td>\n",
       "      <td>2279.0</td>\n",
       "      <td>codingknob</td>\n",
       "      <td>2279.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>9927711</td>\n",
       "      <td>2012-03-29 14:42:42</td>\n",
       "      <td>1</td>\n",
       "      <td>1659</td>\n",
       "      <td>Reading csv in python pandas and handling bad ...</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Dave31415</td>\n",
       "      <td>914.0</td>\n",
       "      <td>eumiro</td>\n",
       "      <td>104313.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54</th>\n",
       "      <td>10003171</td>\n",
       "      <td>2012-04-03 23:59:41</td>\n",
       "      <td>1</td>\n",
       "      <td>404</td>\n",
       "      <td>What is an efficient way in pandas to do summa...</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LmW.</td>\n",
       "      <td>486.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>59</th>\n",
       "      <td>10027719</td>\n",
       "      <td>2012-04-05 11:28:00</td>\n",
       "      <td>0</td>\n",
       "      <td>500</td>\n",
       "      <td>Installing Pandas with Python 2.5 on Windows</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>JamesS</td>\n",
       "      <td>191.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>10393447</td>\n",
       "      <td>2012-05-01 04:12:13</td>\n",
       "      <td>0</td>\n",
       "      <td>130</td>\n",
       "      <td>Scope gotcha when dynamically adding methods i...</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Chris Billington</td>\n",
       "      <td>424.0</td>\n",
       "      <td>Ignacio Vazquez-Abrams</td>\n",
       "      <td>513959.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          id         creationdate  score  viewcount  \\\n",
       "10   8273092  2011-11-25 18:39:02      1       2333   \n",
       "46   9927711  2012-03-29 14:42:42      1       1659   \n",
       "54  10003171  2012-04-03 23:59:41      1        404   \n",
       "59  10027719  2012-04-05 11:28:00      0        500   \n",
       "77  10393447  2012-05-01 04:12:13      0        130   \n",
       "\n",
       "                                                title  answercount  \\\n",
       "10                      python: pandas install errors            2   \n",
       "46  Reading csv in python pandas and handling bad ...            3   \n",
       "54  What is an efficient way in pandas to do summa...            2   \n",
       "59       Installing Pandas with Python 2.5 on Windows            1   \n",
       "77  Scope gotcha when dynamically adding methods i...            2   \n",
       "\n",
       "    commentcount  favoritecount        quest_name  quest_rep  \\\n",
       "10             0            NaN        codingknob     2279.0   \n",
       "46             0            2.0         Dave31415      914.0   \n",
       "54             1            NaN              LmW.      486.0   \n",
       "59             0            NaN            JamesS      191.0   \n",
       "77             0            NaN  Chris Billington      424.0   \n",
       "\n",
       "                  ans_name   ans_rep  \n",
       "10              codingknob    2279.0  \n",
       "46                  eumiro  104313.0  \n",
       "54            Wes McKinney   43310.0  \n",
       "59            Wes McKinney   43310.0  \n",
       "77  Ignacio Vazquez-Abrams  513959.0  "
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "criteria = so['answercount'] > so['score']\n",
    "so[criteria].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In one line, the above would have looked like this:\n",
    "\n",
    "```Python\n",
    "so[so['answercount'] > so['score']]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Almost never use `.iloc` with boolean selection\n",
    "First, remember that **`.iloc`** uses INTEGER location to make its selections. \n",
    "\n",
    "You will rarely use **`.loc`** to do boolean selection and almost always use *just the indexing operator* or **`.loc`**. To see why, let's try and run a simple boolean selection to find all the rows that have more than 100,000 views."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "ename": "NotImplementedError",
     "evalue": "iLocation based boolean indexing on an integer type is not available",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNotImplementedError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-51-9f2c31befcfa>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mso\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0miloc\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mso\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'viewcount'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m100000\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m   1371\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1372\u001b[0m             \u001b[0mmaybe_callable\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcom\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_apply_if_callable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mobj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1373\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getitem_axis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmaybe_callable\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1374\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1375\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_is_scalar_access\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_getitem_axis\u001b[0;34m(self, key, axis)\u001b[0m\n\u001b[1;32m   1812\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1813\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mis_bool_indexer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1814\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_has_valid_type\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1815\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getbool_axis\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0maxis\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1816\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/pandas/core/indexing.py\u001b[0m in \u001b[0;36m_has_valid_type\u001b[0;34m(self, key, axis)\u001b[0m\n\u001b[1;32m   1660\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'index'\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mIndex\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1661\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mkey\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mindex\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minferred_type\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'integer'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1662\u001b[0;31m                     raise NotImplementedError(\"iLocation based boolean \"\n\u001b[0m\u001b[1;32m   1663\u001b[0m                                               \u001b[0;34m\"indexing on an integer type \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1664\u001b[0m                                               \"is not available\")\n",
      "\u001b[0;31mNotImplementedError\u001b[0m: iLocation based boolean indexing on an integer type is not available"
     ]
    }
   ],
   "source": [
    "so.iloc[so['viewcount'] > 100000]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `NotImplementedError`\n",
    "The pandas developers have not decided to boolean selection (with a Series) for **`.iloc`** so it does not work. You can however convert the Series to a list or a NumPy array as a workaround.\n",
    "\n",
    "Let's save our Series to a variable and double-check its type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.series.Series"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "criteria = so['viewcount'] > 100000\n",
    "type(criteria)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's grab the underlying NumPy array with the **`values`** attribute and pass it to **`.iloc`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>7837722</td>\n",
       "      <td>2011-10-20 14:46:14</td>\n",
       "      <td>201</td>\n",
       "      <td>223746</td>\n",
       "      <td>What is the most efficient way to loop through...</td>\n",
       "      <td>8</td>\n",
       "      <td>3</td>\n",
       "      <td>115.0</td>\n",
       "      <td>Muppet</td>\n",
       "      <td>1563.0</td>\n",
       "      <td>Nick Crawford</td>\n",
       "      <td>2779.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>10373660</td>\n",
       "      <td>2012-04-29 16:10:35</td>\n",
       "      <td>199</td>\n",
       "      <td>207980</td>\n",
       "      <td>Converting a Pandas GroupBy object to DataFrame</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>90.0</td>\n",
       "      <td>saveenr</td>\n",
       "      <td>2421.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>10457584</td>\n",
       "      <td>2012-05-05 00:00:12</td>\n",
       "      <td>72</td>\n",
       "      <td>117890</td>\n",
       "      <td>Redefining the Index in a Pandas DataFrame object</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>19.0</td>\n",
       "      <td>nitin</td>\n",
       "      <td>2945.0</td>\n",
       "      <td>Avaris</td>\n",
       "      <td>21067.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>10665889</td>\n",
       "      <td>2012-05-19 14:11:42</td>\n",
       "      <td>144</td>\n",
       "      <td>179896</td>\n",
       "      <td>How to take column-slices of dataframe in pandas</td>\n",
       "      <td>7</td>\n",
       "      <td>3</td>\n",
       "      <td>65.0</td>\n",
       "      <td>cpa</td>\n",
       "      <td>988.0</td>\n",
       "      <td>Ted Petrou</td>\n",
       "      <td>10426.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106</th>\n",
       "      <td>10715965</td>\n",
       "      <td>2012-05-23 08:12:31</td>\n",
       "      <td>340</td>\n",
       "      <td>408347</td>\n",
       "      <td>add one row in a pandas.DataFrame</td>\n",
       "      <td>15</td>\n",
       "      <td>3</td>\n",
       "      <td>89.0</td>\n",
       "      <td>PhE</td>\n",
       "      <td>1988.0</td>\n",
       "      <td>fred</td>\n",
       "      <td>2342.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           id         creationdate  score  viewcount  \\\n",
       "8     7837722  2011-10-20 14:46:14    201     223746   \n",
       "75   10373660  2012-04-29 16:10:35    199     207980   \n",
       "81   10457584  2012-05-05 00:00:12     72     117890   \n",
       "100  10665889  2012-05-19 14:11:42    144     179896   \n",
       "106  10715965  2012-05-23 08:12:31    340     408347   \n",
       "\n",
       "                                                 title  answercount  \\\n",
       "8    What is the most efficient way to loop through...            8   \n",
       "75     Converting a Pandas GroupBy object to DataFrame            5   \n",
       "81   Redefining the Index in a Pandas DataFrame object            2   \n",
       "100   How to take column-slices of dataframe in pandas            7   \n",
       "106                  add one row in a pandas.DataFrame           15   \n",
       "\n",
       "     commentcount  favoritecount quest_name  quest_rep       ans_name  ans_rep  \n",
       "8               3          115.0     Muppet     1563.0  Nick Crawford   2779.0  \n",
       "75              0           90.0    saveenr     2421.0   Wes McKinney  43310.0  \n",
       "81              1           19.0      nitin     2945.0         Avaris  21067.0  \n",
       "100             3           65.0        cpa      988.0     Ted Petrou  10426.0  \n",
       "106             3           89.0        PhE     1988.0           fred   2342.0  "
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = criteria.values\n",
    "so.iloc[a].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can make simultaneous column selection as well with integers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>answercount</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>Nick Crawford</td>\n",
       "      <td>2779.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75</th>\n",
       "      <td>5</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>2</td>\n",
       "      <td>Avaris</td>\n",
       "      <td>21067.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>7</td>\n",
       "      <td>Ted Petrou</td>\n",
       "      <td>10426.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106</th>\n",
       "      <td>15</td>\n",
       "      <td>fred</td>\n",
       "      <td>2342.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     answercount       ans_name  ans_rep\n",
       "8              8  Nick Crawford   2779.0\n",
       "75             5   Wes McKinney  43310.0\n",
       "81             2         Avaris  21067.0\n",
       "100            7     Ted Petrou  10426.0\n",
       "106           15           fred   2342.0"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so.iloc[a, [5, 10, 11]].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I don't think I have ever used **`.iloc`** for boolean selection as its not implemented for Series. I added because it's one of the three main indexers in pandas and it's important to know that it's not used much at all for boolean selection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `.loc` and `[]` work the same on a Series for boolean selection\n",
    "Boolean selection will work identically for **`.loc`** as it does with *just the indexing operator* on a Series. Both the indexers do row selection when passed a boolean Series. Since Series don't have columns, the two indexers are identical in this situation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "s = so['score']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8      201\n",
       "17     136\n",
       "75     199\n",
       "100    144\n",
       "106    340\n",
       "Name: score, dtype: int64"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s[s > 100].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8      201\n",
       "17     136\n",
       "75     199\n",
       "100    144\n",
       "106    340\n",
       "Name: score, dtype: int64"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.loc[s > 100].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Summary\n",
    "\n",
    "* **Boolean Indexing** or **Boolean Selection** is the selection of a subset of a Series/DataFrame based on the values themselves and not the row/column labels or integer location\n",
    "* Boolean selection is used to answer common queries like \"find all the female engineers with a salary over 150k/year\"\n",
    "* To do boolean selection, you first create a sequence of True/False values and pass it to a DataFrame/Series indexer\n",
    "* Each row of data is kept or discarded\n",
    "* The indexing operators are **overloaded** - change functionality depending on what is passed to them\n",
    "* Typically, you will first create a boolean Series with one of the 6 comparison operators\n",
    "* You will pass this boolean series to one of the indexers to make your selection\n",
    "* Use the **`isin`** method to test for multiple equalities in the same column\n",
    "* Use **`isnull`** to find all rows with missing values in a particular column\n",
    "* Can use the **`between`** Series method to test whether Series values are within a range\n",
    "* You can create complex criteria with the **and** (**`&`**), **or** (**`|`**), and **not** (**`~`**) logical operators\n",
    "* When you have multiple conditions in a single line, you must wrap each expression with a parentheses\n",
    "* If you have complex criteria, think about storing each set of criteria into its own variable (i.e. don't do everything in one line)\n",
    "* If you are only selecting rows, then you will almost always use *just the indexing operator*\n",
    "* If you are simultaneously doing boolean selection on the rows and selecting column labels then you will use **`.loc`**\n",
    "* You will almost never use **`.iloc`** to do boolean selection\n",
    "* Boolean selection works the same for Series as it does for DataFrames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# More to the story\n",
    "Believe or not, there is still more to the story when it comes to boolean selection. We will cover more advanced topics in part 7.\n",
    "\n",
    "# Exercises\n",
    "Boolean selection is difficult at first and the syntax is somewhat clunky. It will take some time to master. These questions will start easy and progressively become more difficult."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data for exercises\n",
    "We will use two datasets for these exercises. The stack overflow dataset which you have seen above and the employee dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>creationdate</th>\n",
       "      <th>score</th>\n",
       "      <th>viewcount</th>\n",
       "      <th>title</th>\n",
       "      <th>answercount</th>\n",
       "      <th>commentcount</th>\n",
       "      <th>favoritecount</th>\n",
       "      <th>quest_name</th>\n",
       "      <th>quest_rep</th>\n",
       "      <th>ans_name</th>\n",
       "      <th>ans_rep</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5486226</td>\n",
       "      <td>2011-03-30 12:26:50</td>\n",
       "      <td>4</td>\n",
       "      <td>2113</td>\n",
       "      <td>Rolling median in python</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>yueerhu</td>\n",
       "      <td>125.0</td>\n",
       "      <td>Mike Pennington</td>\n",
       "      <td>26995.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5515021</td>\n",
       "      <td>2011-04-01 14:50:44</td>\n",
       "      <td>8</td>\n",
       "      <td>7015</td>\n",
       "      <td>Compute a compounded return series in Python</td>\n",
       "      <td>3</td>\n",
       "      <td>6</td>\n",
       "      <td>7.0</td>\n",
       "      <td>Jason Strimpel</td>\n",
       "      <td>3301.0</td>\n",
       "      <td>Mike Pennington</td>\n",
       "      <td>26995.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5558607</td>\n",
       "      <td>2011-04-05 21:13:50</td>\n",
       "      <td>2</td>\n",
       "      <td>7392</td>\n",
       "      <td>Sort a pandas DataMatrix in ascending order</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Jason Strimpel</td>\n",
       "      <td>3301.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6467832</td>\n",
       "      <td>2011-06-24 12:31:45</td>\n",
       "      <td>9</td>\n",
       "      <td>13056</td>\n",
       "      <td>How to get the correlation between two timeser...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>user814005</td>\n",
       "      <td>117.0</td>\n",
       "      <td>Wes McKinney</td>\n",
       "      <td>43310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7577546</td>\n",
       "      <td>2011-09-28 01:58:38</td>\n",
       "      <td>9</td>\n",
       "      <td>2488</td>\n",
       "      <td>Using pandas, how do I subsample a large DataF...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>Uri Laserson</td>\n",
       "      <td>958.0</td>\n",
       "      <td>HYRY</td>\n",
       "      <td>54137.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        id         creationdate  score  viewcount  \\\n",
       "0  5486226  2011-03-30 12:26:50      4       2113   \n",
       "1  5515021  2011-04-01 14:50:44      8       7015   \n",
       "2  5558607  2011-04-05 21:13:50      2       7392   \n",
       "3  6467832  2011-06-24 12:31:45      9      13056   \n",
       "4  7577546  2011-09-28 01:58:38      9       2488   \n",
       "\n",
       "                                               title  answercount  \\\n",
       "0                           Rolling median in python            3   \n",
       "1       Compute a compounded return series in Python            3   \n",
       "2        Sort a pandas DataMatrix in ascending order            2   \n",
       "3  How to get the correlation between two timeser...            1   \n",
       "4  Using pandas, how do I subsample a large DataF...            1   \n",
       "\n",
       "   commentcount  favoritecount      quest_name  quest_rep         ans_name  \\\n",
       "0             4            1.0         yueerhu      125.0  Mike Pennington   \n",
       "1             6            7.0  Jason Strimpel     3301.0  Mike Pennington   \n",
       "2             0            1.0  Jason Strimpel     3301.0     Wes McKinney   \n",
       "3             0            7.0      user814005      117.0     Wes McKinney   \n",
       "4             0            5.0    Uri Laserson      958.0             HYRY   \n",
       "\n",
       "   ans_rep  \n",
       "0  26995.0  \n",
       "1  26995.0  \n",
       "2  43310.0  \n",
       "3  43310.0  \n",
       "4  54137.0  "
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "so = pd.read_csv('../../data/stackoverflow_qa.csv')\n",
    "so.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>UNIQUE_ID</th>\n",
       "      <th>POSITION_TITLE</th>\n",
       "      <th>DEPARTMENT</th>\n",
       "      <th>BASE_SALARY</th>\n",
       "      <th>RACE</th>\n",
       "      <th>EMPLOYMENT_TYPE</th>\n",
       "      <th>GENDER</th>\n",
       "      <th>EMPLOYMENT_STATUS</th>\n",
       "      <th>HIRE_DATE</th>\n",
       "      <th>JOB_DATE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>ASSISTANT DIRECTOR (EX LVL)</td>\n",
       "      <td>Municipal Courts Department</td>\n",
       "      <td>121862.0</td>\n",
       "      <td>Hispanic/Latino</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Female</td>\n",
       "      <td>Active</td>\n",
       "      <td>2006-06-12</td>\n",
       "      <td>2012-10-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>LIBRARY ASSISTANT</td>\n",
       "      <td>Library</td>\n",
       "      <td>26125.0</td>\n",
       "      <td>Hispanic/Latino</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Female</td>\n",
       "      <td>Active</td>\n",
       "      <td>2000-07-19</td>\n",
       "      <td>2010-09-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>POLICE OFFICER</td>\n",
       "      <td>Houston Police Department-HPD</td>\n",
       "      <td>45279.0</td>\n",
       "      <td>White</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Male</td>\n",
       "      <td>Active</td>\n",
       "      <td>2015-02-03</td>\n",
       "      <td>2015-02-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>ENGINEER/OPERATOR</td>\n",
       "      <td>Houston Fire Department (HFD)</td>\n",
       "      <td>63166.0</td>\n",
       "      <td>White</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Male</td>\n",
       "      <td>Active</td>\n",
       "      <td>1982-02-08</td>\n",
       "      <td>1991-05-25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>ELECTRICIAN</td>\n",
       "      <td>General Services Department</td>\n",
       "      <td>56347.0</td>\n",
       "      <td>White</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Male</td>\n",
       "      <td>Active</td>\n",
       "      <td>1989-06-19</td>\n",
       "      <td>1994-10-22</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   UNIQUE_ID               POSITION_TITLE                     DEPARTMENT  \\\n",
       "0          0  ASSISTANT DIRECTOR (EX LVL)    Municipal Courts Department   \n",
       "1          1            LIBRARY ASSISTANT                        Library   \n",
       "2          2               POLICE OFFICER  Houston Police Department-HPD   \n",
       "3          3            ENGINEER/OPERATOR  Houston Fire Department (HFD)   \n",
       "4          4                  ELECTRICIAN    General Services Department   \n",
       "\n",
       "   BASE_SALARY             RACE EMPLOYMENT_TYPE  GENDER EMPLOYMENT_STATUS  \\\n",
       "0     121862.0  Hispanic/Latino       Full Time  Female            Active   \n",
       "1      26125.0  Hispanic/Latino       Full Time  Female            Active   \n",
       "2      45279.0            White       Full Time    Male            Active   \n",
       "3      63166.0            White       Full Time    Male            Active   \n",
       "4      56347.0            White       Full Time    Male            Active   \n",
       "\n",
       "    HIRE_DATE    JOB_DATE  \n",
       "0  2006-06-12  2012-10-13  \n",
       "1  2000-07-19  2010-09-18  \n",
       "2  2015-02-03  2015-02-03  \n",
       "3  1982-02-08  1991-05-25  \n",
       "4  1989-06-19  1994-10-22  "
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "employee = pd.read_csv('../../data/employee.csv')\n",
    "employee.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tip!\n",
    "Append the **`head`** method at the end of your statements to prevent long output as was done above.\n",
    "\n",
    "# Solutions\n",
    "Make sure you check your answers with the [solutions notebook](./Solutions.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1\n",
    "<span  style=\"color:green; font-size:16px\">Find all the questions that have exactly 5 answers</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 2\n",
    "<span  style=\"color:green; font-size:16px\">Find all the questions that have less than 10 views</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 3\n",
    "<span  style=\"color:green; font-size:16px\">Find all the questions where the person asking it is the same as the person answering it</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4\n",
    "<span  style=\"color:green; font-size:16px\">Find all the questions that don't have an accepted answer, but have a score of more than 100</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 5\n",
    "<span  style=\"color:green; font-size:16px\">Find all the questions where the reputation of the person asking the question is higher than the person answering it. Then find the percentage of times this happens</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 6\n",
    "<span  style=\"color:green; font-size:16px\">Find all the questions where the number of answers is between 5 and 10 inclusive, and the number of views is less than 1,000.</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 7\n",
    "<span  style=\"color:green; font-size:16px\">Find the inverse of exercise 6. Verify your results by adding the rows of both returned Series to see if it matches the number of rows of the original</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Use the employee data for the rest of the exercises"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>UNIQUE_ID</th>\n",
       "      <th>POSITION_TITLE</th>\n",
       "      <th>DEPARTMENT</th>\n",
       "      <th>BASE_SALARY</th>\n",
       "      <th>RACE</th>\n",
       "      <th>EMPLOYMENT_TYPE</th>\n",
       "      <th>GENDER</th>\n",
       "      <th>EMPLOYMENT_STATUS</th>\n",
       "      <th>HIRE_DATE</th>\n",
       "      <th>JOB_DATE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>ASSISTANT DIRECTOR (EX LVL)</td>\n",
       "      <td>Municipal Courts Department</td>\n",
       "      <td>121862.0</td>\n",
       "      <td>Hispanic/Latino</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Female</td>\n",
       "      <td>Active</td>\n",
       "      <td>2006-06-12</td>\n",
       "      <td>2012-10-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>LIBRARY ASSISTANT</td>\n",
       "      <td>Library</td>\n",
       "      <td>26125.0</td>\n",
       "      <td>Hispanic/Latino</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Female</td>\n",
       "      <td>Active</td>\n",
       "      <td>2000-07-19</td>\n",
       "      <td>2010-09-18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>POLICE OFFICER</td>\n",
       "      <td>Houston Police Department-HPD</td>\n",
       "      <td>45279.0</td>\n",
       "      <td>White</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Male</td>\n",
       "      <td>Active</td>\n",
       "      <td>2015-02-03</td>\n",
       "      <td>2015-02-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>ENGINEER/OPERATOR</td>\n",
       "      <td>Houston Fire Department (HFD)</td>\n",
       "      <td>63166.0</td>\n",
       "      <td>White</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Male</td>\n",
       "      <td>Active</td>\n",
       "      <td>1982-02-08</td>\n",
       "      <td>1991-05-25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>ELECTRICIAN</td>\n",
       "      <td>General Services Department</td>\n",
       "      <td>56347.0</td>\n",
       "      <td>White</td>\n",
       "      <td>Full Time</td>\n",
       "      <td>Male</td>\n",
       "      <td>Active</td>\n",
       "      <td>1989-06-19</td>\n",
       "      <td>1994-10-22</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   UNIQUE_ID               POSITION_TITLE                     DEPARTMENT  \\\n",
       "0          0  ASSISTANT DIRECTOR (EX LVL)    Municipal Courts Department   \n",
       "1          1            LIBRARY ASSISTANT                        Library   \n",
       "2          2               POLICE OFFICER  Houston Police Department-HPD   \n",
       "3          3            ENGINEER/OPERATOR  Houston Fire Department (HFD)   \n",
       "4          4                  ELECTRICIAN    General Services Department   \n",
       "\n",
       "   BASE_SALARY             RACE EMPLOYMENT_TYPE  GENDER EMPLOYMENT_STATUS  \\\n",
       "0     121862.0  Hispanic/Latino       Full Time  Female            Active   \n",
       "1      26125.0  Hispanic/Latino       Full Time  Female            Active   \n",
       "2      45279.0            White       Full Time    Male            Active   \n",
       "3      63166.0            White       Full Time    Male            Active   \n",
       "4      56347.0            White       Full Time    Male            Active   \n",
       "\n",
       "    HIRE_DATE    JOB_DATE  \n",
       "0  2006-06-12  2012-10-13  \n",
       "1  2000-07-19  2010-09-18  \n",
       "2  2015-02-03  2015-02-03  \n",
       "3  1982-02-08  1991-05-25  \n",
       "4  1989-06-19  1994-10-22  "
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "employee = pd.read_csv('../../data/employee.csv')\n",
    "employee.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 8\n",
    "<span  style=\"color:green; font-size:16px\">Find all the **`Black or African American`** females that work in the **`Houston Police Department-HPD`**</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 9\n",
    "<span  style=\"color:green; font-size:16px\">Find the females that have a salary over 100,000 OR males with salary under 50,000</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 10\n",
    "<span  style=\"color:green; font-size:16px\">Find the females in the following departments with salary over 60,000 (Parks & Recreation, Solid Waste Management, Fleet Management Department, Library)  </span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 11\n",
    "<span  style=\"color:green; font-size:16px\">Find all the males with salary over 100,000. Return only the race, gender and salary columns</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 12\n",
    "<span  style=\"color:green; font-size:16px\">Select all salaries as a Series in a separate variable. From this series select all salaries under 25,000</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 13\n",
    "<span  style=\"color:green; font-size:16px\">Get the same exact result as exercise 11, but make your selection from the employee DataFrame. Use only a single line of code</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# your code here"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
